Digital stethoscope for counting coughs, and applications thereof

ABSTRACT

Embodiments disclosed herein improve digital stethoscopes and their application and operation. A first method detects of a respiratory abnormality using a convolution. A second method counts coughs for a patient. A third method predicts a respiratory event based on a detected trend. A fourth method forecasts characteristics of a future respiratory event. In a fifth embodiment, a base station is provided for a digital stethoscope.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.16/933,430, filed Jul. 20, 2020, which is a continuation of co-pendingU.S. application Ser. No. 16/659,371, filed Oct. 21, 2019 (now U.S. Pat.No. 10,750,976), both of which are incorporated in their entirety byreference.

TECHNICAL FIELD

An embodiment relates generally to a digital stethoscope, and morespecifically a digital stethoscope for detecting a respiratoryabnormality and a method of operation thereof.

BACKGROUND

Respiratory illnesses such as asthma, chronic obstructive pulmonarydisease (COPD), bronchitis, emphysema, and pneumonia affect manyindividuals. The ability to quickly detect and forecast the onset ofthese conditions, including possible life threatening events associatedwith these conditions, is of vital importance to those affected.Generally, diagnosis of these respiratory illnesses involves a doctorthat listens to patient's breathing using a stethoscope. A stethoscopeis an acoustic medical device for auscultation. It typically has a smalldisc-shaped resonator that is placed against the skin, and a tubeconnected to two earpieces. However, these traditional stethoscopes areprone to error and require a doctor to be present and to make thediagnosis. The need for a doctor makes daily monitoring for theseconditions impractical.

A number of patents and applications have been filed that attempt todeal with these issues. U.S. Pat. No. 9,848,848 describes a digitalstethoscope that uses a number of audio filters to control noise andreduce the possibility of error. U.S. Patent Pub. No. 2018/0317876describes using a classification system, such as a binary support vectormachine, to distinguish between those noises that are normal from thosethat are abnormal.

However, a number of limitations still exist in the art. For example,there is a need to improve real-time performance of the classificationalgorithm to allow it to be executed in real time and locally on adevice that exists at the patient's home. There may be a need to improvethe ability to forecast future respiratory future respiratory events.There may be a need to catalog data collected from in-home stethoscopes,while protecting a patient's privacy interest. Currently, aclassification system may be able to predict whether a noise is normalor abnormal, but cannot predict a severity of a future respiratory eventor the characteristics of that respiratory event. Methods, devices, andsystems are needed to address these issues.

SUMMARY

Embodiments disclosed herein improve digital stethoscopes and theirapplication and operation. In a first embodiment, a method detects of arespiratory abnormality using a convolution. The convolution may improveperformance, particularly in situations where the detection should occurin real-time. The method involves receiving, from a microphone, anauditory signal. An auditory spectrogram is generated based on theauditory signal. A convolution procedure is performed on the auditoryspectrogram to generate one or more convolution values. The convolutionvalues represent a compressed version of a portion of the auditoryspectrogram, and the convolution procedure is trained to generate one ormore features for detecting the respiratory abnormality. One or moreweights trained for detection of the respiratory abnormality are appliedto the convolution values. A classification value is generated based onthe application of the weights to the convolution values. Theclassification value indicates whether the auditory signal includes therespiratory abnormality.

In a second embodiment, a method counts coughs for a patient. The methodinvolves receiving from a microphone of a digital stethoscope anauditory signal over a period of time. An auditory spectrogram isgenerated based on the auditory signal. A control unit of the digitalstethoscope analyzes the auditory spectrogram to determine whether theauditory signal represents a cough. The control unit of the digitalstethoscope tracks the cough in a cough log. Based on the cough log, acommunication unit of the digital stethoscope transmits to a cloud-basedservice a message indicating a number of coughs tracked over the periodof time for storage on a remote server.

In a third embodiment, a method predicts a respiratory event based on adetected trend. The method involves receiving, from a microphone of adigital stethoscope, a first noise signal at a first time. The firstnoise signal is analyzed to determine a first severity score indicatinghow severe a first respiratory event of a patient captured in the firstnoise signal is. A second noise signal captured at a second time isreceived from the microphone of the digital stethoscope. The secondnoise signal is analyzed to determine a second severity score indicatinghow severe a second respiratory event of the patient captured in thesecond noise signal is. Finally, a prediction is generated indicating alikelihood of a third respiratory event occurring for the patient at afuture time based on applying a first weight to the first severity scoreand a second weight to the second severity score. The first and secondweights are trained based on a history of respiratory events.

In a fourth embodiment, a method forecasts characteristics of a futurerespiratory event. The method involves receiving, from a microphone of adigital stethoscope, a first noise signal. The first noise signalcaptures a first respiratory event at a first time. A first featurespace representing the first noise signal is generated. The firstfeature space is encoded into a first convolution vector using aconvolution procedure. A second noise signal is received, from themicrophone of the digital stethoscope. The second noise signal capturesa second respiratory event at a second time. A second feature spacerepresenting the second noise signal is generated. The second featurespace is encoded into a second convolution vector using the convolutionprocedure. A predicted convolution vector is generated based on thefirst and second feature spaces. Finally, the predicted convolutionvector is decoded into a predicted feature space representing a soundmade by the future respiratory event.

In a fifth embodiment, a base station is provided for a digitalstethoscope. The base station includes a housing configured to acceptthe digital stethoscope. The base station includes a wireless chargingunit, located within the housing, configured to charge the digitalstethoscope when it rests on the exterior of the housing. The wirelesscharging unit is further configured to detect when the digitalstethoscope becomes detached from the housing. A communication unit iscoupled to the wireless charging unit. The communication unit isconfigured to receive a noise signal from the digital stethoscope whenthe wireless charging unit detects the digital stethoscope becomingdetached from the housing and to communicate the respiratory event to acloud-based service. Finally, a control unit is coupled to thecommunication unit and is configured to analyze the noise signal for arespiratory event.

Certain embodiments of the invention have other steps or elements inaddition to or in place of those mentioned above. The steps or elementswill become apparent to those skilled in the art from a reading of thefollowing detailed description when taken with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and form a partof the specification, illustrate embodiments of the present disclosureand, together with the description, further serve to explain theprinciples of the disclosure and to enable a person skilled in the artsto make and use the embodiments.

FIG. 1 is an exemplary digital stethoscope and base station of a systemfor detecting a respiratory abnormality in an embodiment of the presentinvention.

FIG. 1A is an exemplary architecture of the digital stethoscope in anembodiment of the present invention.

FIG. 1B is an exemplary architecture of the base station in anembodiment of the present invention.

FIG. 2A is a network diagram of the system for detecting the respiratoryabnormality including further components of the digital stethoscope andbase station in an embodiment of the present invention.

FIG. 2B is an exemplary control flow of the system for detecting therespiratory abnormality in an embodiment of the present invention.

FIG. 2C is a further exemplary architecture of the digital stethoscopein an embodiment of the present invention.

FIG. 2D is a further exemplary architecture of the base station in anembodiment of the present invention.

FIG. 3 is an exemplary second control flow for detection of therespiratory abnormality in an embodiment of the present invention.

FIG. 4A is an exemplary depiction of the functioning of a receivermodule and a subtraction module in an embodiment of the presentinvention.

FIG. 4B is an exemplary depiction of the functioning of a filter moduleand the generation of an auditory spectrogram in an embodiment of thepresent invention.

FIG. 4C is an exemplary depiction of the functioning of a convolutionneural network module in an embodiment of the present invention.

FIG. 5A is an exemplary feature map used to predict a respiratory eventor respiratory condition in the future in an embodiment of the presentinvention.

FIG. 5B is an exemplary depiction of an LSTM process used to predict athird respiratory event occurring for the patient at a future time in anembodiment of the present invention.

FIG. 6 is an exemplary third control flow for forecastingcharacteristics of a future respiratory event or respiratory conditionin an embodiment of the present invention.

FIG. 7 is an exemplary depiction of the third control flow forforecasting characteristics of a future respiratory event or respiratorycondition in an embodiment of the present invention.

FIG. 8 is an exemplary method of operating the computing system in anembodiment of the present invention.

FIG. 9 is a further exemplary method of operating the system in anembodiment of the present invention.

FIG. 10 is a further exemplary method of operating the system in anembodiment of the present invention.

FIG. 11 is a further exemplary method of operating the system in anembodiment of the present invention.

FIG. 12 depicts exemplary spectrograms representing breathing patternsin an embodiment of the present invention.

FIG. 13 depicts an exemplary auditory spectrogram in an embodiment ofthe present invention.

FIG. 14 depicts an exemplary feature space in an embodiment of thepresent invention.

In the drawings, like reference numbers generally indicate identical orsimilar elements. Additionally, generally, the left-most digit(s) of areference number identifies the drawing in which the reference numberfirst appears.

DETAILED DESCRIPTION Digital Stethoscope and Base Station

FIG. 1 shows an exemplary digital stethoscope 110 and base station 118of a system 100 for detecting a respiratory abnormality in an embodimentof the present invention. The respiratory abnormality refers toirregularities in the respiration patterns of a patient. The respiratoryabnormality can indicate the onset of a respiratory condition in apatient, for example a respiratory disease such as asthma, chronicobstructive pulmonary disease (COPD), bronchitis, emphysema, pneumonia,or a combination thereof. The respiratory abnormality can be indicatedby the occurrence of a respiratory event, such as a large number ofcoughs within a period of time by a patient, a wheeze, a crackle, or acough that has a sound frequency outside an audible frequency expectedfrom a patient, or a combination thereof. The system 100 can use thedigital stethoscope 110 and the base station 118 to detect therespiratory abnormality, or predict a respiratory event or respiratorycondition in the future.

The digital stethoscope 110 is an acoustic device for detecting andanalyzing noises from a patient's body. The patient can be, for example,a human or an animal. The noises, from the patient's body can be forexample a cough, a wheeze, a crackle, a breathing pattern, a heartbeat,a chest motion representing a patient's respiratory cycle, or acombination thereof.

The digital stethoscope 110 can further generate information based onthe detection, amplification, and analysis of the noises. For example,in one embodiment, the digital stethoscope 110 can generate a valuerepresenting or classifying the noises detected.

In one embodiment, the classification can include classifications suchas “normal” or “abnormal.” “Normal” refers to the classification ofsounds falling within an expected frequency range to be heard from thepatient. “Abnormal” refers to the classification of sounds fallingoutside an expected frequency range to be heard from the patient.Classification can be done by analyzing the noises, by for example,filtering, comparing, processing, or a combination thereof, the noises,against threshold values, stored values, acoustic models, machinelearned trained data, machine learning processes, or a combinationthereof, and putting the noises into categories, for example “normal” or“abnormal” based on the noises being within a range of frequenciesexpected to be heard from the patient. The collection, filtering,comparison, and classification of the noises by the digital stethoscope110 will be discussed further below.

The digital stethoscope 110 can include one or more components. Forexample, in one embodiment, the digital stethoscope 110 can include adisplay unit 102, one or more microphones 106, and a first housing 108.The display unit 102 can be any graphical user interface such as adisplay, a projector, a video screen, a touch screen, or any combinationthereof that can present information detected or generated by thedigital stethoscope 110 for visualization by a user of the system 100.The display unit 102 can enable the visual presentation of informationdetected or generated by the digital stethoscope 110.

For example, in one embodiment, the display unit 102 can enable thevisual presentation of the noises detected, by for example, displaying aplot of the sound frequencies detected over time, displaying a decibellevel of the sounds detected, or displaying a value or visual indicatorrepresenting the classification of the noises generated, for example“normal” or “abnormal.” In one embodiment, if the digital stethoscope110 classifies a noise as being “abnormal,” the display unit 102 candisplay an indicator, such as a red colored light, or a messageindicating that the noise is “abnormal.” Alternatively, if the digitalstethoscope 110 classifies the noise as being “normal,” the display unit102 can display an indicator, such as a green colored light, or amessage indicating that the noise is “normal.”

The display unit 102 can further present other information generated bythe digital stethoscope 110, such as a power level indicator indicatinghow much power the digital stethoscope has, a volume indicatorindicating the volume level of output noises being output by the digitalstethoscope 110, or a network connectivity indicator indicating whetherthe digital stethoscope 110 is connected to a device or computer networksuch as a wireless communication network or wired communication network.The aforementioned information are merely exemplary of the types ofinformation that the display unit 102 can display, and are not meant tobe limiting.

In one embodiment, the display unit 102 can further include one or morebuttons 126 that can be used by the user of the system 100 to enableinteraction with the digital stethoscope 110. For example, the buttons126 can provide functionality such as powering the digital stethoscope110 on or off or enable the digital stethoscope 110 to start or stoprecording the noises.

In one embodiment, the digital stethoscope 110 can further include oneor more microphones 106A and B. The microphones 106A and B enable thedigital stethoscope 110 to detect and convert the noises into electricalsignals for processing by the digital stethoscope 110, or a furtherdevice such as the base station 118. Microphone 106A is mounted on aperimeter side of stethoscope 110 to detect noises external to thepatient's body. The noises originating from external to the patient'sbody can be for example background noise, white noise, or a combinationthereof. Microphone 106B may be mounted on a side reverse of display 102and may detect noises originating from the patient's body.

The microphones 106A and B can be standalone devices or can be arrangedin an array configuration, where the microphones 106 operate in tandemto detect the noises. In one embodiment, each microphone in the arrayconfiguration 104 can serve a different purpose. For example, eachmicrophone in the array configuration 104 can be configured to detectand convert into electrical signals the noises at different frequenciesor within different frequency ranges such that each of the microphones106 can be configured to detect specific noises. The noises detected bythe microphones 106 can be used to generate the values for classifyingthe noises as “normal” or “abnormal,” and can be further used to predictthe respiratory event or respiratory condition in the future.

The digital stethoscope 110 can further have a first housing 108enclosing the components of the digital stethoscope 110. The firsthousing 108 can separate components of the digital stethoscope 110contained within from other components external to the first housing108. For example, the first housing 108 can be a case, a chassis, a box,or a console. In one embodiment, for example, the components of thedigital stethoscope 110 can be contained within the first housing 108.In another embodiment, some components of the digital stethoscope 110can be contained within the first housing 108 while other components,such as the display 102, the microphones 106, the buttons 126, or acombination thereof, can be accessible external to the first housing108. The aforementioned are merely examples of components that can becontained in or on the first housing 108 and are not meant to belimiting. Further discussion of other components of the digitalstethoscope 110 will be discussed below.

The system 100 can further include a base station 118. The base station118 is a special purpose computing device that enables computation andanalysis of the noises obtained by the digital stethoscope 110 in orderto detect the respiratory abnormality, or to predict the respiratoryevent or respiratory condition in the future. The base station 118 canprovide additional or higher performance processing power compared tothe digital stethoscope 110. In one embodiment, the base station 118 canwork in conjunction with the digital stethoscope 110 to detect, amplify,adjust, and analyze noises from a patient's body by, for example,providing further processing, storage, or communication capabilities tothe digital stethoscope 110. In another embodiment, the base station 118can work as a standalone device to detect, amplify, adjust, and analyzenoises to detect the respiratory abnormality, or to predict therespiratory event or respiratory condition in the future.

The base station 118 can analyze of the noises captured by stethoscope110. For example, in one embodiment, the base station 118 can generatevalues classifying the noises detected as “normal” or “abnormal.” Thecollection, filtering, comparison, and classification of the noises bythe base station 118 will be discussed further below.

The base station 118 can include one or more components. For example, inone embodiment, the base station 118 can include a charging pad 114, oneor more air quality sensors 116, a contact sensor 120, and a secondhousing 112. The charging pad 114 can enable the electric charging ofthe digital stethoscope 110, through inductive charging where anelectromagnetic field is used to transfer energy between the chargingpad 114 and a further device, such as the digital stethoscope 110, usingelectromagnetic induction.

In one embodiment, the charging pad 114 can enable electric charging ofthe digital stethoscope 110 upon detecting contact or coupling, via thecontact sensor 120, between the digital stethoscope 110 and the chargingpad 114. For example, in one embodiment, if the digital stethoscope 110is coupled to the charging pad 114 by physical placement of the digitalstethoscope 110 on the charging pad 114, the contact sensor 120 candetect a weight or an electromagnetic signal produced by the digitalstethoscope 110 on the charging pad 114, and upon sensing the weight orthe electromagnetic signal enable the induction process to transferenergy between the charging pad 114 and the digital stethoscope 110.

In another embodiment, if the digital stethoscope 110 is coupled to thecharging pad 114 by placing the digital stethoscope 110 in proximity ofthe charging pad 114 without physically placing the digital stethoscope110 on the charging pad 114, the contact sensor 120 can detect anelectric current or a magnetic field from one or more components of thedigital stethoscope 110 and enable the induction process to transferenergy between the charging pad 114 and the digital stethoscope 110.

The contact sensor 120 is a device that senses mechanical orelectromagnetic contact and gives out signals when it does so. Thecontact sensor 120 can be, for example, a pressure sensor, a forcesensor, strain gauges, piezoresistive/piezoelectric sensors, capacitivesensors, elastoresistive sensors, torque sensors, linear force sensors,an inductor, other tactile sensors, or a combination thereof configuredto measure a characteristic associated with contact or coupling betweenthe digital stethoscope 110 and the charging pad 114. Accordingly, thecontact sensor 120 can output a contact measure 122 that represents aquantified measure, for example, a measured force, a pressure, anelectromagnetic force, or a combination thereof corresponding to thecoupling between the digital stethoscope 110 and the charging pad 114.For example, the contact measure 122 can detect one or more force orpressure readings associated with forces applied by the digitalstethoscope 110 on the charging pad 114. The contact measure 122 canfurther detect one or more electric current or magnetic field readingsassociated with placing the digital stethoscope 110 in proximity of thecharging pad 114.

In one embodiment, the base station 118 can further include one or moreair quality sensors 116. The air quality sensors 116 are devices thatdetect and monitor the presence of air pollution in a surrounding area.Air pollution refers to the presence of or introduction into the air ofa substance which has harmful or poisonous effects on the patient'sbody. For example, the air quality sensors 116 can detect the presenceof particulate matter or gases such as ozone, carbon monoxide, sulfurdioxide, nitrous oxide, or a combination thereof that can be poisonousto the patient's body, and in particular poisonous to the patient'srespiratory system.

In one embodiment, based on the air quality sensors 116 detecting thepresence of air pollution, the base station 118 can determine whetherthe amount of air pollution poses a health risk to the patient by, forexample, comparing the levels of air pollution to a pollution threshold124 to determine whether the levels of air pollution in the surroundingarea of the base station 118 pose a health risk to the patient. Thepollution threshold 124 refers to a pre-determined level for particulatematter or gases measured in micrograms per cubic meter (μg/m3), partsper million (ppm), or parts per billion (ppb), that if exceeded poses ahealth risk to the patient

For example, in one embodiment, if the air quality sensors 116 detectthe presence of sulfur dioxide above 75 ppb in the air surrounding thebase station 118, the base station 118 can determine that the airpollution in the surrounding area poses a health risk to the patient.The detection of air pollution can further be used for detecting therespiratory abnormality or to predict the respiratory event orrespiratory condition in the future in the patient by allowing thesystem 100 to determine what factors are contributing to the “normal” or“abnormal” classification of the noises, or what factors arecontributing to the data detected and generated by the system 100 whichcan be used to predict a respiratory event or respiratory condition inthe future.

The base station 118 can further have a second housing 112 enclosing thecomponents of the base station 118. The second housing 112 can separatecomponents of the base station 118 contained within, from othercomponents external to the second housing 112. For example, the secondhousing 112 can be a case, a chassis, a box, or a console. In oneembodiment, for example, the components of the base station 118 can becontained within the second housing 112. In another embodiment, somecomponents of the base station 118 can be contained within the secondhousing 112 while other components, such as the charging pad 114 or theair quality sensors 116 can be accessible external to the second housing112. The aforementioned are merely examples of components that can becontained in or on the second housing 112 and are not meant to belimiting. Further discussion of other components of the base station 118will be discussed below.

Referring now to FIG. 1A, therein is shown an exemplary architecture ofthe digital stethoscope 110 in an embodiment. In various embodiments,the digital stethoscope 110 can include, alternatively or additionally:

-   -   a grip ring 140 located around a first upper portion 142 of the        first housing 108 which provides a gripping surface for a user        of the system 100 to hold the digital stethoscope 110;    -   a glass lens 144 of the display unit 102, which protects the        display components, such as for example liquid crystal displays        (LCD) of the display unit 102. The glass lens 144 can sit on top        of a housing gasket 146, which stabilizes and holds the glass        lens 144;    -   a display housing unit 148, on which the housing gasket 146 sits        and which contains the components of the display unit 102, such        as for example the LCDs;    -   a flex backing 150 which on which the display housing 148 sits        and which provides stability for the display housing 148;    -   a flex assembly 152, on which the flex backing 150 sits and        which provides stability for the flex backing 150;    -   a retainer clip 154 which holds the flex assembly 152 in place;    -   a battery housing 156, to which a battery board 158 can couple,        and which can hold battery components of the digital stethoscope        110;    -   a first printed circuit board assembly 164, which can hold the        circuitry, including any processors, memory components, active        and passive components, or a combination thereof, of the digital        stethoscope 110;    -   one or more first screws 162 that couples the first printed        circuit board assembly 164 to the other components of the        digital stethoscope 110;    -   an audio jack 168 to allow output of noise signals detected by        the digital stethoscope 110;    -   a microphone assembly 170, on which the microphones 106 can be        housed;    -   components such as an O-ring 172 and one or more coils 166 that        couple the microphone assembly 170 to the first printed circuit        board assembly 164.    -   a first bottom portion 174 of the first housing 108 on which the        microphone assembly 170 sits;    -   a diaphragm membrane 182 which forms the bottom surface of the        digital stethoscope 110, and which is coupled to the first        bottom portion 174 of the first housing 108 with one or more        second screws 176 and one or more washers 178; and    -   a diaphragm ring 180 coupled to the diaphragm membrane 182,        which provides a gripping surface for the first bottom portion        174 of the digital stethoscope 110, such that the digital        stethoscope 110 does not slip when placed on a surface.

The aforementioned components are merely exemplary and represent oneembodiment of the digital stethoscope 110.

Referring now to FIG. 1B, that figure illustrates an exemplaryarchitecture of the base station 118 in an embodiment of the presentinvention. In various embodiments, the base station 118 can include,alternatively or additionally:

-   -   a second upper portion 134 of the second housing 112;    -   a second printed circuit board assembly 130 which can hold the        circuitry, including any processors, memory components, active        and passive components, or a combination thereof, of the base        station 118;    -   one or more third screws 132 that couples the second printed        circuit board assembly 130 to the second upper portion 134 of        the second housing 112 via one or more second connectors 126;    -   one or more coils 128, coupled to the second printed circuit        board assembly 130, which can detect the weight or the        electromagnetic signal produced by the digital stethoscope 110        on the base station 118;    -   a second bottom portion 136 of the second housing 112, which        forms the bottom surface of the base station 118; and    -   one or more bumpers 138 to cover and protect the third screws        132.

The aforementioned components are merely exemplary and represent oneembodiment of the base station 118.

Referring now to FIG. 2A, that figure shows a network diagram of thesystem 100 for detecting the respiratory abnormality including furthercomponents of the digital stethoscope 110 and base station 118 in anembodiment of the present invention. FIG. 2A shows an embodiment wherethe digital stethoscope 110 is connected to the base station 118 througha device or computer network, such as a wireless or wired network, via acommunication path 246. FIG. 2A further shows an embodiment where aremote server 242 is connected to the digital stethoscope 110 and thebase station 118 via the communication path 246.

The remote server 242 can provide additional or higher performanceprocessing power compared to the digital stethoscope 110 and the basestation 118. In one embodiment, the remote server 242 can work inconjunction with the digital stethoscope 110, the base station 118, or acombination thereof to analyze the detected noises. For example, in oneembodiment, the remote server 242 can provide some or all of theprocessing power of the system 100 to process the information detectedor generated by the digital stethoscope 110, the base station 118, or acombination thereof.

In one embodiment, the remote server 242 can further provide additionalstorage capabilities to the digital stethoscope 110, the base station118, or a combination thereof by enabling the storage of informationdetected or generated by the digital stethoscope 110, the base station118, or a combination thereof, and provide access to the digitalstethoscope 110, the base station 118, or a combination thereof of theinformation for later use. For example, in one embodiment, the remoteserver 242 can store the noises detected by the digital stethoscope 110,the base station 118, or a combination thereof for later retrieval bythe digital stethoscope 110, the base station 118, or a combinationthereof.

In one embodiment, the remote server 242 can further provideinformation, such as the pre-determined threshold values, stored values,acoustic models, machine learned trained data, machine learningprocesses, configuration data, or a combination thereof to the digitalstethoscope 110, the base station 118, or a combination thereof to allowthe digital stethoscope 110, the base station 118, or a combinationthereof to perform some or all of their functions. For example, theremote server 242 can store and provide access to the pollutionthreshold 124 to the base station 118 to allow the base station 118 toperform the computations and comparisons needed to detect and monitorthe presence of air pollution in the surrounding area.

In one embodiment, the remote server 242 can further provideconfiguration data, such as software updates, including updated acousticmodels, machine learned trained data, machine learning processes, or acombination thereof to the digital stethoscope 110, the base station118, or a combination thereof to allow the digital stethoscope 110, thebase station 118, or a combination thereof to perform computations andanalysis in order to determine the classifications of the noises and todetect the respiratory abnormality or to predict the respiratory eventor respiratory condition in the future.

The remote server 242 can be any of a variety of centralized ordecentralized computing devices. For example, the remote server 242 canbe a laptop computer, a desktop computer, grid-computing resources, avirtualized computing resource, cloud computing resources, routers,switches, peer-to-peer distributed computing devices, a server, or acombination thereof. The remote server 242 can be centralized in asingle room, distributed across different rooms, distributed acrossdifferent geographical locations, or embedded within atelecommunications network. The remote server 242 can couple with thecommunication path 246 to communicate with the digital stethoscope 110,the base station 118, or a combination thereof.

The communication path 246 can span and represent a variety of networksand network topologies. For example, the communication path 246 caninclude wireless communication, wired communication, opticalcommunication, ultrasonic communication, or a combination thereof. Forexample, satellite communication, cellular communication, Bluetooth,Infrared Data Association standard (IrDA), wireless fidelity (WiFi), andworldwide interoperability for microwave access (WiMAX) are examples ofwireless communication that can be included in the communication path246. Cable, Ethernet, digital subscriber line (DSL), fiber optic lines,fiber to the home (FTTH), and plain old telephone service (POTS) areexamples of wired communication that can be included in thecommunication path 246. Further, the communication path 246 can traversea number of network topologies and distances. For example, thecommunication path 246 can include direct connection, personal areanetwork (PAN), local area network (LAN), metropolitan area network(MAN), wide area network (WAN), or a combination thereof.

Also for illustrative purposes, the system 100 is shown with the digitalstethoscope 110, the base station 118, and the remote server 242 as endpoints of the communication path 246, although it is understood that thesystem 100 can have a different partition between the digitalstethoscope 110, the base station 118, the remote server 242, and thecommunication path 246. For example, the digital stethoscope 110, thebase station 118, and the remote server 242, or a combination thereofcan also function as part of the communication path 246.

In one embodiment, the digital stethoscope 110 can include furthercomponents including a first control unit 210, a first storage unit 218,a first communication unit 204, the first display unit 102, themicrophone array 104, a first location unit 256, and a battery 288. Thefirst control unit 210 can include a first control interface 212. Thefirst control unit 210 can execute a first software 224 to provide theintelligence of the system 100. The first control unit 210 can beimplemented in a number of different ways. For example, the firstcontrol unit 210 can be a first processor 214, a first fieldprogrammable gate array (FPGA) 216, an application specific integratedcircuit (ASIC), an embedded processor, a microprocessor, a hardwarecontrol logic, a hardware finite state machine (FSM), a digital signalprocessor (DSP), or a combination thereof.

The first control interface 212 can be used for communication betweenthe first control unit 210 and other components of the digitalstethoscope 110. The first control interface 212 can also be used forcommunication that is external to the digital stethoscope 110. The firstcontrol interface 212 can receive information from the other componentsof the digital stethoscope 110 or from external sources, or can transmitinformation to the other components of the digital stethoscope 110 or toexternal destinations. The external sources and the externaldestinations refer to sources and destinations external to the digitalstethoscope 110. The first control interface 212 can be implemented indifferent ways and can include different implementations depending onwhich components or external units are being interfaced with the firstcontrol interface 212. For example, the first control interface 212 canbe implemented with a pressure sensor, an inertial sensor, amicroelectromechanical system (MEMS), optical circuitry, waveguides,wireless circuitry, wireline circuitry such as a bus interface, anapplication programming interface (API), or a combination thereof.

The first storage unit 218 can store the first software 224 to providethe intelligence of the system 100. For illustrative purposes, the firststorage unit 218 is shown as a single element, although it is understoodthat the first storage unit 218 can be a distribution of storageelements. Also for illustrative purposes, the system 100 is shown withthe first storage unit 218 as a single hierarchy storage system,although it is understood that the system 100 can have the first storageunit 218 in a different configuration. For example, the first storageunit 218 can be formed with different storage technologies forming amemory hierarchal system including different levels of caching, mainmemory, rotating media, or off-line storage. The first storage unit 218can be a volatile memory, a nonvolatile memory, an internal memory, anexternal memory, or a combination thereof. For example, the firststorage unit 218 can be a nonvolatile storage such as non-volatilerandom access memory (NVRAM), Flash memory, disk storage, or a volatilestorage such as static random access memory (SRAM) or a first dynamicrandom access memory (DRAM) 254.

The first storage unit 218 can include a first storage interface 220.The first storage interface 220 can be used for communication betweenthe first storage unit 218 and other components of the digitalstethoscope 110. The first storage interface 220 can also be used forcommunication that is external to the digital stethoscope 110. The firststorage interface 220 can receive information from the other componentsof the digital stethoscope 110 or from external sources, or can transmitinformation to the other components or to external destinations. Thefirst storage interface 220 can include different implementationsdepending on which components or external units are being interfacedwith the first storage unit 218. The first storage interface 220 can beimplemented with technologies and techniques similar to theimplementation of the first control interface 212.

The first communication unit 204 can enable external communication toand from the digital stethoscope 110. For example, the firstcommunication unit 204 can permit the digital stethoscope 110 tocommunicate with the remote server 242, the base station 118, anattachment, such as a peripheral device, and the communication path 246.The first communication unit 204 can also function as a communicationhub allowing the digital stethoscope 110 to function as part of thecommunication path 246 and not be limited to be an end point or terminalunit to the communication path 246. The first communication unit 204 caninclude active and passive components, such as microelectronics or anantenna, for interaction with the communication path 246. The firstcommunication unit 204 can further have circuitry, such as a firstBluetooth circuit 206, a wireless fidelity (WiFi) circuit, a Near FieldCommunication (NFC) circuit, or a combination thereof for interactionwith the communication path 246.

The first communication unit 204 can include a first communicationinterface 208. The first communication interface 208 can be used forcommunication between the first communication unit 204 and othercomponents of the digital stethoscope 110. The first communicationinterface 208 can receive information from the other components of thedigital stethoscope 110 or from external sources, or can transmitinformation to the other components or to external destinations. Thefirst communication interface 208 can include different implementationsdepending on which components are being interfaced with the firstcommunication unit 204. The first communication interface 208 can beimplemented with technologies and techniques similar to theimplementation of the first control interface 212. The firstcommunication unit 204 can couple with the communication path 246 tosend information to the remote server 242, the base station 118, or acombination thereof.

The first location unit 256 can generate location information, currentheading, and current speed and acceleration of the digital stethoscope110, as examples. The first location unit 256 can be implemented in manyways. For example, the first location unit 256 can include components,such as a GPS receiver, an inertial navigation system, a cellular-towerlocation system, a pressure location system, an accelerometer 226, agyroscope, or any combination thereof. The components can be used inconjunction with other components of the digital stethoscope 110 todetect movements or the location of the patient. For example, in oneembodiment, the accelerometer 226 can be used to detect whether thepatient or a portion of the patient's body is moving, for example todetect a chest motion representing a patient's respiratory cycle. Forexample, if the digital stethoscope 110 is placed on the patient'schest, the accelerometer 226 can detect the up and down movement of thepatient's chest to determine the frequency and speed at which thepatient's chest is moving. In one embodiment, the location unit 256 canfurther be used to detect the physical location of the digitalstethoscope 110, such as the geographic location. For example, thelocation unit 256 can detect the location of the digital stethoscope 110by using the accelerometer 226 in conjunction with the other components,and determine that a patient may be physically moving from onegeographic location to another.

In one embodiment, the detection of the movement can generateinformation and data regarding the noises detected by the digitalstethoscope 110. For example, if rapid movement of the digitalstethoscope 110 is detected and the digital stethoscope 110 detects highfrequency noises at the same time, the digital stethoscope 110 candetermine that there is high likelihood that the patient may be movingexcessively and can further determine that some of the noise generatedby the movement should be removed or filtered because it constitutesunwanted background noise. The information detected as a result of themovement can further be used by the digital stethoscope 110 to adjustand analyze the noises, by for example using the information to amplifycertain frequencies of the noises that are desired or necessary for thedetection of the respiratory abnormality or to reduce or suppresscertain frequencies of the noises that are unwanted and not necessaryfor the detection of the respiratory abnormality. As a result, thedigital stethoscope 110 can reduce noise and amplify the noises tofurther improve the accuracy when detecting the respiratory abnormality,or to predict a respiratory event or respiratory condition at a futuretime.

The first location unit 256 can include a first location interface 258.The first location interface 258 can be used for communication betweenthe first location unit 256 and other components of the digitalstethoscope 110. The first location interface 258 can also be used forcommunication that is external to the digital stethoscope 110. The firstlocation interface 258 can be implemented with technologies andtechniques similar to the implementation of the first control interface212.

The battery 288 is the power source for the digital stethoscope 110. Inone embodiment, the battery 288 can include one or more electrochemicalcells with external connections provided to power the digitalstethoscope 110. The electrochemical cells can include primary ornon-rechargeable cells, secondary or rechargeable cells, or acombination thereof. For example, in one embodiment, the electrochemicalcells can include secondary cells that can be charged wirelessly usingelectromagnetic induction. In one embodiment, electrochemical cells caninclude primary cells such as alkaline batteries, lithium batteries, ora combination thereof.

In one embodiment, the base station 118 can include further componentsincluding a second control unit 236, a second storage unit 248, a secondcommunication unit 228, and sensor unit 202. The second control unit 236can include a second control interface 238. The second control unit 236can execute a second software 252 to provide the intelligence of thesystem 100. The second software 252 can operate independently or inconjunction with the first software 224. The second control unit 236 canprovide additional performance compared to the first control unit 210.The second control unit 236 can be a second processor 240, a secondfield programmable gate array (FPGA) 244, an application specificintegrated circuit (ASIC), an embedded processor, a microprocessor, ahardware control logic, a hardware finite state machine (FSM), a digitalsignal processor (DSP), or a combination thereof.

The second control unit 236 can include a second control interface 238.The second control interface 238 can be used for communication betweenthe second control unit 236 and other components of the base station118. The second control interface 238 can also be used for communicationthat is external to the base station 118. The second control interface238 can receive information from the other components of the basestation 118 or from external sources, or can transmit information to theother components of the base station 118 or to external destinations.The external sources and the external destinations refer to sources anddestinations external to the base station 118. The second controlinterface 238 can be implemented in different ways and can includedifferent implementations depending on which components or externalunits are being interfaced with the second control interface 238. Forexample, the second control interface 238 can be implemented with apressure sensor, an inertial sensor, a microelectromechanical system(MEMS), optical circuitry, waveguides, wireless circuitry, wirelinecircuitry such as a bus interface, an application programming interface(API), or a combination thereof.

The second storage unit 248 can be sized to provide additional storagecapacity to supplement the first storage unit 218. For illustrativepurposes, the second storage unit 248 is shown as a single element,although it is understood that the second storage unit 248 can be adistribution of storage elements. Also for illustrative purposes, thesystem 100 is shown with the second storage unit 248 as a singlehierarchy storage system, although it is understood that the system 100can have the second storage unit 248 in a different configuration. Forexample, the second storage unit 248 can be formed with differentstorage technologies forming a memory hierarchal system includingdifferent levels of caching, main memory, rotating media, or off-linestorage. The second storage unit 248 can be a volatile memory, anonvolatile memory, an internal memory, an external memory, or acombination thereof. For example, the second storage unit 248 can be anonvolatile storage such as non-volatile random access memory (NVRAM),Flash memory, disk storage, or a volatile storage such as static randomaccess memory (SRAM) or a second dynamic random access memory (DRAM)222.

The second storage unit 248 can include a second storage interface 250.The second storage interface 250 can be used for communication betweenthe second storage unit 248 and other components of the base station118. The second storage interface 250 can also be used for communicationthat is external to the base station 118. The second storage interface250 can receive information from the other components of the basestation 118 or from external sources, or can transmit information to theother components or to external destinations. The second storageinterface 250 can include different implementations depending on whichcomponents or external units are being interfaced with the secondstorage unit 248. The second storage interface 250 can be implementedwith technologies and techniques similar to the implementation of thesecond control interface 238.

The second communication unit 228 can enable external communication toand from the base station 118. For example, the second communicationunit 228 can permit the base station 118 to communicate with the digitalstethoscope 110, the remote server 242, an attachment, such as aperipheral device, and the communication path 246. The secondcommunication unit 228 can also function as a communication hub allowingthe base station 118 to function as part of the communication path 246and not be limited to be an end point or terminal unit to thecommunication path 246. The second communication unit 228 can includeactive and passive components, such as microelectronics or an antenna,for interaction with the communication path 246. The secondcommunication unit 228 can further have circuitry, such as a secondBluetooth circuit 230, a wireless fidelity (WiFi) circuit, a Near FieldCommunication (NFC) circuit, an internet-of-things (IoT) modem 232, or acombination thereof for interaction with the communication path 246.

The second communication unit 228 can include a second communicationinterface 234. The second communication interface 234 can be used forcommunication between the second communication unit 228 and othercomponents of the base station 118. The second communication interface234 can receive information from the other components of the basestation 118 or from external sources, or can transmit information to theother components or to external destinations. The second communicationinterface 234 can include different implementations depending on whichcomponents are being interfaced with the second communication unit 228.The second communication interface 234 can be implemented withtechnologies and techniques similar to the implementation of the secondcontrol interface 238.

The sensor unit 202 can enable the base station 118 to obtain one ormore sensed readings 286 used to perform one or more of the basestation's 118 functions. The sensed readings 286 can include informationor data obtained by the sensor unit 202, the purpose of which is todetect events or changes in the environment of the base station 118 andto send the information to components of the base station 118, thedigital stethoscope 110, the remote server 242, external devices such asa peripheral device, or a combination thereof to facilitate thefunctionality of the system 100. The sensed readings 286 can include thecontact measure 122 or the amount of air pollution in the surroundingarea. In one embodiment, the sensor unit 202 can include the air qualitysensors 116, the contact sensor 120, or a combination thereof.

The sensor unit 202 can include a sensor unit interface 260. The sensorunit interface 260 can be used for communication between the sensor unit202 and other components of the base station 118. The sensor unitinterface 260 can also be used for communication that is external to thebase station 118. The sensor unit interface 260 can receive informationfrom the other components of the base station 118 or from externalsources, or can transmit information to the other components of the basestation 118 or to external destinations. The sensor unit interface 260can include different implementations depending on which components ofthe base station 118 or external units are being interfaced with thesensor unit 202. The sensor unit interface 260 can be implemented withtechnologies and techniques similar to the implementation of the secondcontrol interface 238.

Referring now to FIG. 2B, that figure shows an exemplary control flow200 of the system 100 for detecting the respiratory abnormality in anembodiment. In one embodiment, one or more auditory signals 262 can bedetected by the digital stethoscope 110 using the microphones 106. Theauditory signals 262 can include the noises from the patient's body orfrom external to the patient's body. By way of example, FIG. 2B depictstwo auditory signals 262, where “C1” represents a cough, crackle, orwheeze originating from the patient's body and “N1” represents a noisesignal resulting from noises generated external to the patient's body,for example background noise from a patient's environment. In oneembodiment, the auditory signals 262 can be saved in a sound file 290,such as a .wav file generated by the digital stethoscope 110. In oneembodiment, once the auditory signals 262 are detected and saved in thesound file 290, the digital stethoscope 110 can amplify, adjust, andanalyze the auditory signals 262 in the sound file 290, by for example,subtracting, suppressing, or filtering N1 to leave only C1. In oneembodiment, the sound file 290 can be stored on the first storage unit218, the second storage unit 248, or the remote server 242.

In one embodiment, the digital stethoscope 110, the base station 118, ora combination thereof can further generate a configuration file 264 andstore the information from detecting, amplifying, adjusting, andanalyzing the auditory signals 262 in the configuration file 264. Theconfiguration file 264 is a computer file, such as a text file, thatcontains the information from detecting, amplifying, adjusting, andanalyzing the auditory signals 262. In one embodiment, the configurationfile 264 can be stored in and accessed from the first storage unit 218,the second storage unit 248, or the remote server 242. In oneembodiment, the configuration file 264 can include information such as aclassification value 266 indicating whether the auditory signals 262indicate a respiratory sound, noise, or auditory tone that is classifiedas “normal” or “abnormal.” The configuration file 264 can furtherinclude a timestamp 268 indicating when the auditory signals 262 weredetected, an accelerometer data 272 obtained from the accelerometer 226indicating movement of the patient, a cough count 274 indicating thenumber of coughs detected by the digital stethoscope 110 over a periodof time, or a combination thereof.

In one embodiment, the configuration file 264 can further includeinformation about devices of the system 100, such as a serial number 270of the digital stethoscope 110 to indicate what device detected theauditory signals 262. The aforementioned are merely exemplary and otherinformation detected and generated by the digital stethoscope 110 can bestored in the configuration file 264.

In one embodiment, the configuration file 264 can be sent to one or moredevices of the system 100. For example, the configuration file 264 canbe sent to the digital stethoscope 110, the base station 118, the remoteserver 242, or a combination thereof to be used in performingadjustments or analysis of the auditory signals 262, or to detect arespiratory abnormality or to predict a respiratory event or respiratorycondition in the future. For example, in one embodiment where thedigital stethoscope 110, the base station 118, the remote server 242, ora combination thereof are performing some or all of the processing ofthe auditory signals 262, the digital stethoscope 110, the base station118, the remote server 242, or a combination thereof can receive theauditory signals 262 via the sound file 290, and further information viathe configuration file 264, parse the two files, extract informationfrom the two files, and perform processing based on the informationcontained in the two files.

Referring now to FIG. 2C, therein is shown a further exemplaryarchitecture of the digital stethoscope 110 in an embodiment of thepresent invention. FIG. 2C shows an embodiment where the digitalstethoscope 110 includes the first control unit 210 and the firststorage unit 218. The first control unit 210 can include the firstprocessor 214 and the first FPGA 216. The first storage unit 218 caninclude the first DRAM 254. The first processor 214 and the first FPGA216 can be coupled using the first control interface 212. The firststorage unit 218 can be coupled to the first control unit 210 using thefirst storage interface 220. For example, in one embodiment, the firstDRAM 254 can be coupled, via the first storage interface 220 to thefirst processor 214.

In one embodiment, the first processor 214, the first FPGA 216, and thefirst DRAM 254 can work in conjunction to process the auditory signals262 detected by the microphones 106 or the array configuration 104. Inone embodiment, the first processor 214 can act as a controller andcontrol the coordination, communications, scheduling, and transfers ofdata between the first FPGA 216, the first DRAM 254, or other componentsof the digital stethoscope 110. For example, in one embodiment, thefirst processor 214 can receive the auditory signals 262 from themicrophones 106, and transfer the auditory signals 262 to the first FPGA216 for further processing. In one embodiment, once the first FPGA 216has completed its operations, the first FPGA 216 can transfer the outputor data generated as a result of its operations back to the firstprocessor 214, which can further transfer the output or data to thefirst DRAM 254 for storage. In one embodiment, the first processor 214can further enable the generation of the configuration file 264. In oneembodiment, the first processor 214 can further enable the generation ofthe sound file 290.

In one embodiment, the first FPGA 216 can perform the processing of theauditory signals 262. The first FPGA 216 can include one or more logicblocks, including one or more reconfigurable logic gates, that can bepre-programmed or configured to perform calculations or computations onthe auditory signals 262, and to generate output or data to detect therespiratory abnormality, or to predict a respiratory event orrespiratory condition in the future. The first FPGA 216 can, forexample, have its logic blocks preconfigured with threshold values,stored values, acoustic models, machine learned trained data, machinelearning processes, configuration data, or a combination thereof thatcan be used to perform the processing on the auditory signals 262, theresult of which is to detect the respiratory abnormality, or to predictthe respiratory event or respiratory condition in the future.

For example, in one embodiment the first FPGA 216 can be preconfiguredwith a machine learning process, for example a convolutional neuralnetwork model, which can have one or more weights 276 associatedtherewith. The weights 276 refer to values, parameters, thresholds, or acombination thereof that act as filters in the machine learning processand represent particular features of the sounds, noises, and acoustictones of a respiratory abnormality, respiratory event, respiratorycondition, or a combination thereof. The weights 276 can be iterativelyadjusted based on machine learned trained data.

Continuing with the example, the first FPGA 216 can, in one embodiment,use the machine learning process, including the weights 276 to detectwhether the auditory signals 262 contain a sound, noise, or acoustictone indicative of a respiratory abnormality, or whether the auditorysignals 262 are indicative of a respiratory event or respiratorycondition in the future. Further discussion of the processing done bythe first FPGA 216 will be discussed below.

Referring now to FIG. 2D, therein is shown a further exemplaryarchitecture of the base station 118 in an embodiment of the presentinvention. FIG. 2D shows an embodiment where the base station 118includes the second control unit 236, the sensor unit 202, the secondcommunication unit 228, and a wireless charging unit 278. The secondcontrol unit 236 can include the second processor 240 and the secondFPGA 244. The sensor unit 202 can include the contact sensor 120 and theair quality sensors 116. The second communication unit 228 can includethe IoT modem 232 and the second Bluetooth circuit 230. The secondBluetooth circuit 230 can further include a real time audio circuit 280and a data transfer circuit 282. The real time audio circuit 280 and thedata transfer circuit 282 can enable the base station 118 to connect tomultiple devices simultaneously over a Bluetooth connection. Forexample, in one embodiment, the real time audio circuit 280 can enable aBluetooth connection to the digital stethoscope 110 to send or receivethe auditory signals 262 or the sound file 290 containing the auditorysignals 262, and the data transfer circuit 282 can enable simultaneousBluetooth connection to a further device, such as a mobile phone 284 tocommunicate outputs or data generated by the base station 118 as aresult of processing the auditory signals 262. In one embodiment, theIoT modem 232 can further be used to communicate outputs or datagenerated by the base station 118 to a further device, for example theremote server 242. In one embodiment, the IoT modem 232 can further beused to receive configuration data, such as software updates, includingupdated acoustic models, machine learned trained data, machine learningprocesses, firmware, or a combination thereof from the remote server242. In one embodiment, the base station 118 can further communicate thesoftware updates to the digital stethoscope 110 using the secondBluetooth circuit 230.

The second processor 240 and the second FPGA 244 can be coupled usingthe second control interface 238. The second communication unit 228 cancouple to the second control unit 236 using the second communicationinterface 234. The sensor unit 202 can couple to the second control unit236 using the sensor unit interface 260. The sensor unit 202 can coupleto the wireless charging unit 278 using the sensor unit interface 260.

In one embodiment, the second processor 240 can act as a controller andcontrol the coordination, communications, scheduling, and transfers ofdata between the second FPGA 244 and other components of the basestation 118. For example, in one embodiment, the second processor 240can receive the auditory signals 262 from the digital stethoscope 110via the second communication unit 228, and transfer the auditory signals262 to the second FPGA 244 for further processing. In one embodiment,once the second FPGA 244 has completed its operations, the second FPGA244 can transfer the output or data generated as a result of itsoperations back to the second processor 240, which can further transferthe output or data to other components of the base station 118. Forexample, the second processor 240 can further transfer the output ordata to the second communication unit 228 for transfer to the remoteserver 242, the mobile device 284, the digital stethoscope 110, or acombination thereof. The mobile device 284 can be a device associatedwith a user of the system 100 that the base station 118 can use tocommunicate the output or data generated by the base station 118, thedigital stethoscope 110, the remote server 242, or a combination thereofto a user of the system 100. The mobile device 284 can be, for example,a mobile phone, a smart phone, a tablet, a laptop, or a combinationthereof.

In one embodiment, the second processor 240 can further generate theconfiguration file 264 and store values, variables, configuration data,time stamps, or data detected or generated therein. In one embodiment,the second processor 240 can further generate the sound file 290 forstoring the auditory signals 262.

In one embodiment, the second FPGA 244 can perform the processing of theauditory signals 262. The second FPGA 244 can include one or more logicblocks, including one or more reconfigurable logic gates, that can bepre-programmed or configured to perform calculations or computations onthe auditory signals 262, and to generate output or data generated todetect the respiratory abnormality, or to predict a respiratory event orrespiratory condition in the future. The second FPGA 244 can, forexample, have its logic blocks preconfigured with threshold values,stored values, acoustic models, machine learned trained data, machinelearning processes, configuration data, or a combination thereof thatcan be used to perform the processing on the auditory signals 262, theresult of which is to detect the respiratory abnormality, or to predictthe respiratory event or respiratory condition in the future.

For example, in one embodiment the second FPGA 244 can be preconfiguredwith a machine learning process, for example a convolutional neuralnetwork model, which can have one or more weights 276 as shown in FIG.2C, associated therewith. In another embodiment, the second FPGA 244 canbe preconfigured with a machine learning process, for example a longshort term memory (LSTM) network model, which can have one or moreweights 276 associated therewith. In one embodiment, the second FPGA 244can be work with the remote server 242 to implement the machine learningprocess, for example the convolutional neural network model, or the LSTMnetwork model, wherein the second FPGA 244 and the remote server 242 candivide the processing needed to perform the computations done by themachine learning process.

Continuing with the example, the second FPGA 244 can, in one embodiment,use the machine learning process to detect whether the auditory signals262 contain a sound, noise, or acoustic tone indicative of a respiratoryabnormality. In another embodiment, the second FPGA 244 can use themachine learning process to predict a respiratory event or respiratorycondition in the future using the auditory signals 262. Furtherdiscussion of the processing done by the second FPGA 244 will bediscussed below.

The wireless charging unit 278 can enable the electric charging of thedigital stethoscope 110, through inductive charging by, for example,generating the electromagnetic field used to transfer energy between thecharging pad 114 of FIG. 1 , and a further device, such as the digitalstethoscope 110 using electromagnetic induction. The wireless chargingunit 278 can include the processors, active and passive components,circuitry, control logic, or a combination thereof to enable theinductive charging. In one embodiment, the wireless charging unit 278can couple to the contact sensor 120 to enable the inductive charging.For example, in one embodiment, if the contact sensor 120 detectscontact or coupling between the digital stethoscope 110 and the chargingpad 114, the contact sensor 120 can generate the contact measure 122 ofFIG. 1 , which can be sent to the wireless charging unit 278. Thewireless charging unit 278 upon receiving the contact measure 122 candetermine that a coupling between the digital stethoscope 110 and thecharging pad 114 has occurred and can activate the base station's 118processors, active and passive components, circuitry, control logic, ora combination thereof to generate the electromagnetic field and begintransferring energy between the charging pad 114 and the digitalstethoscope 110. In one embodiment, the wireless charging unit 278 canfurther power off the base station 118 during the time period in whichit is charging the digital stethoscope 110 by, for example, generating asignal to the second processor 240 that charging is taking place andthat the components of the base station 118 should be in an off or idlemode during the time period.

In one embodiment, the wireless charging unit 278 can further enable theactivation of the base station 118 based on determining a termination ofthe coupling between the digital stethoscope 110 and the charging pad114. For example, in one embodiment, the wireless charging unit 278 candetect a termination of the coupling between the digital stethoscope 110and the charging pad 114 based on a change in the contact measure 122.For example, in one embodiment, if the digital stethoscope 110 isremoved from the charging pad 114, the contact sensor 120 can generate acontact measure 122 indicating the removal, and can send the contactmeasure 122 to the wireless charging unit 278. The wireless chargingunit 278 upon receiving the contact measure 122 can determine that thecoupling between the digital stethoscope 110 and the charging pad 114 isno longer present and can send a signal to the second processor 240 toactivate or power up the components of the base station 118, so that thebase station 118 can perform computations and processing on auditorysignals 262, or communicate with further devices such as the digitalstethoscope 110, the mobile device 284, the remote server 242, or acombination thereof.

Convolutional Neural Network to Detect Abnormality

Referring now to FIG. 3 , therein is shown an exemplary second controlflow 300 for detection of the respiratory abnormality in an embodimentof the present invention. For brevity of description, in thisembodiment, the second control flow 300 will be described as beingperformed using the digital stethoscope 110. This description is merelyexemplary and not meant to be limiting. In other embodiments, the secondcontrol flow 300 can be performed using the base station 118, the remoteserver 242, or a combination thereof.

In one embodiment, the second control flow 300 can be implemented withmodules and sub-modules, the components of the digital stethoscope 110,or a combination thereof. In one embodiment, the second control flow 300can include a receiver module 304, a subtraction module 306, a filtermodule 308, and a convolution neural network module 310. In oneembodiment, the receiver module 304 can couple to the subtraction module306. The subtraction module 306 can couple to the filter module 308. Thefilter module 308 can couple to the convolution neural network module310.

The receiver module 304 can enable receiving of one or more signals ordata by the digital stethoscope 110. In one embodiment, the signals ordata can be, for example, the auditory signals 262, the accelerometerdata 272, or a combination thereof, and can be received via themicrophones 106 of FIG. 1 , the accelerometer of FIG. 2A, theconfiguration file 264, the sound file 290, or a combination thereof. Byway of example, FIG. 3 depicts two auditory signals 262, where “C1”represents a cough, crackle, or wheeze originating from the patient'sbody and “N1” represents a noise signal resulting from noises generatedexternal to the patient's body, for example background noise from apatient's environment. FIG. 3 further depicts the accelerometer data272, where “A1” represents the accelerometer data 272. In oneembodiment, C1, N1, and A1, represent time series signals and data, inwhich the signals are measured over a period of time and can berepresented mathematically by one or more time functions 302, in whichthe value of the time functions 302 vary based on time.

In one embodiment, the receiver module 304 can further enable theconversion of the time functions 302 into one or more frequency domainfunctions 330, in which the time functions 302 can be represented withrespect to a frequency component of the signals or data. The frequencydomain functions 330 can be used by the digital stethoscope 118 toperform further calculations or processing of the auditory signals 262,the accelerometer data 272, or a combination thereof to detect therespiratory abnormality. For example, in one embodiment, C1 and N1 canbe represented as a combined time function 302 x(t), where x(t) can berepresented by equation (1) below:

x(t)=y(t)+N(t)  (1)

In equation (1), y(t) represents the time function 302 of C1 and N(t)represents the time function 302 of N1. The receiver module 304 canconvert x(t) into a frequency domain function 330 by, for example,performing a Fourier transformation or Fast Fourier transformation onx(t) such that the frequency domain function 330 is obtained, wherex′(w) in FIG. 3 represents the frequency domain function 330 of x(t). Inone embodiment, the digital stethoscope 110 can then use x′(w) toperform its calculations and computations. Similarly, in one embodiment,the receiver module 304 can convert the acceleration data 272 from atime function 302 to a frequency domain function 330 using similartechniques.

In one embodiment, once the receiver module 304 receives the auditorysignals 262, the accelerometer data 272, or a combination thereof andconverts the signals from a time function 302 to frequency domainfunctions 330, the receiver module 304 can pass control and thefrequency domain functions 330 to the subtraction module 306. Thesubtraction module 306 can enable the adjusting the auditory signals 262received from the receiver module 304 by removing or filtering ofunwanted auditory signals 262, for example a background noise created bythe movement of the patient and picked up by the accelerometer 226, orother noise signals originating external to the patient's body, toobtain a noise free or nearly noise free auditory signals 262. Thesubtraction module 306 can enable the removal or filtering of unwantedauditory signals 262 by implementing a filter to remove unwanted soundfrequencies. In one embodiment, the filter can be implemented usinghardware, software, or a combination thereof. For example, the filtercan be implemented to perform the functions of, or be, a low passfilter, a high pass filter, a bandpass filter, a Butterworth filter, aChebyshev filter, a Bessel filter, a Elliptic filter, or a combinationthereof.

In one embodiment, the subtraction module 306 can implement the filtersuch that the filter performs removal or filtering based on equation (2)below:

x(w)=y(w)−(αβΔN(w))  (2)

In equation (2), x(w) represents the filtered frequency domain function330 of the auditory signals 262, accelerometer data 272, or acombination thereof obtained from the receiver module 304, y(w)represents the frequency domain function 330 of the auditory signals262, accelerometer data 272, or a combination thereof obtained from thereceiver module 304 that should be kept by the digital stethoscope 110,N(w) represents the frequency domain function 330 of the auditorysignals 262, accelerometer data 272, or a combination thereof obtainedfrom the receiver module 304 that should not be kept by the digitalstethoscope 110, and α, β, and Δ are variables that can be integers orwhole numbers used to determine how much of N(w) should be subtractedfrom y(w) such that a low noise or noise free signal remains. In oneembodiment, α, β, and Δ can be varied or adjusted over time based on thenoise conditions in which the digital stethoscope 110 is used. Forexample, in one embodiment, α, β, and Δ can be varied or adjusted every20, 50, or 100 milliseconds. In one embodiment, α, β, and Δ can bevaried or adjusted wherein α, β, and Δ are large values in a high noiseenvironment, such as a noisy office environment, or small values in alow noise environment, such as a quiet office environment. In oneembodiment, α, β, and Δ can be a set of pre-determined values. Inanother embodiment, α, β, and Δ can change dynamically based on changesin the noise levels detected by the digital stethoscope 110. In oneembodiment, α, β, and Δ can be pre-configured values in the first FPGA216 or can be obtained from the base station 118, the remote server 242,or a combination thereof.

In one embodiment, once the subtraction module 306 performs itsadjustment and filtering of the auditory signals 262, the accelerometerdata 272, or a combination thereof, the subtraction module 306 can passcontrol and the filtered frequency domain functions 330 of the auditorysignals 262 to the filter module 308. The filter module 308 enables thegeneration of an auditory spectrogram 312 by further separating orfiltering the filtered frequency domain functions 330 of the auditorysignals 262, accelerometer data 272, or a combination thereof into oneor more frequency bands 336 representing the different audiblefrequencies of the filtered frequency domain functions 330 of theauditory signals 262, the accelerometer data 272, or a combinationthereof.

The auditory spectrogram 312 refers to a two dimensional visualrepresentation of the spectrum of frequencies of the filtered frequencydomain functions 330 of the auditory signals 262, the accelerometer data272, or a combination thereof. The auditory spectrogram 312 can berepresented as a chart, image, matrix, heat map, or other visualdepiction showing a plot of the frequencies and the magnitudesassociated with the frequencies of the filtered frequency domainfunctions 330 of the auditory signals 262, accelerometer data 272, or acombination thereof. The auditory spectrogram 312 can enable thesharpening or amplification of the frequency components filteredfrequency domain functions 330 of the auditory signals 262,accelerometer data 272, or a combination thereof to better depict whatsounds, noises, or acoustic tones are present in the filtered frequencydomain functions 330 of the auditory signals 262, accelerometer data272, or a combination thereof. The auditory spectrogram 312 can be usedin the processing of the filtered frequency domain functions 312 of theauditory signals 262, accelerometer data 272, or a combination thereofto detect the respiratory abnormality, and result in more accuratelydetermining the respiratory abnormality due to the ability of theauditory spectrogram 312 to sharpen and amplify the frequency componentsof the filtered frequency domain functions 330 of the auditory signals262, accelerometer data 272, or a combination thereof.

In one embodiment, the filter module 308 can generate the auditoryspectrogram 312 by implementing one or more cochlear filters 332 whichthe filtered frequency domain functions 330 of the auditory signals 262,accelerometer data 272, or a combination thereof can be passed through,and wherein the cochlear filters 332 are based on a cochlear model 314.The cochlear model 314 refers to the mathematical model orrepresentation of the mechanics of a mammalian cochlea. In oneembodiment, the cochlear model 314 can be implemented using hardware,software, or a combination thereof. For example, the cochlear filters332 can include processors, active and passive components, circuitry,control logic, or a combination thereof implementing the cochlear model314. In one embodiment, the cochlear filters 332 can separate differentaudible frequencies into one or more frequency ranges that a mammaliancochlea can hear.

For example, in one embodiment, the filter module 308 can implement onehundred and twenty-eight (128) cochlear filters 332 representing the onehundred and twenty eight different frequency ranges that can be heard bythe mammalian cochlea. As such, the filter module 308 can mimic thehuman ear. In one embodiment, the output of the filter module 308 basedon further filtering using the cochlear filters 332, the filteredfrequency domain functions 330 of the auditory signals 262,accelerometer data 272, or a combination thereof is the auditoryspectrogram 312. The auditory spectrogram 312 can visually represent thefrequency ranges obtained from further filtering the filtered frequencydomain functions 330 of the auditory signals 262, accelerometer data272, or a combination thereof.

In one embodiment, once the filter module 308 generates the auditoryspectrogram 312, the filter module 308 can pass control and the auditoryspectrogram 312 to the convolution neural network module 310. Theconvolution neural network module 310 can enable the detection of therespiratory abnormality by performing a convolution procedure 316 on theauditory spectrogram 312 and further passing the results to a neuralnetwork 326 for classification as “normal” or “abnormal.” Thedesignation of “normal” or “abnormal” indicates whether the auditorysignals 262, accelerometer data 272, or a combination thereof detectedby the digital stethoscope 110 indicates a respiratory abnormality.

The convolution procedure 316 refers to a mathematical operation on twofunctions to produce a third function that expresses how the shape ofone of the functions is modified by the other function. In oneembodiment, the convolution procedure 316 can be implemented based onequation (3) below:

$\begin{matrix}\begin{matrix}{{{f\left\lbrack {x,y} \right\rbrack}*{g\left\lbrack {x,y} \right\rbrack}} = {\sum\limits_{n_{1} = {- \infty}}^{\infty}{\sum\limits_{n_{2} = {- \infty}}^{\infty}{{f\left\lbrack {n_{1},n_{2}} \right\rbrack} \cdot {g\left\lbrack {{x - n_{1}},{y - n_{2}}} \right\rbrack}}}}} \\{{{f\left\lbrack {x,y} \right\rbrack}*{g\left\lbrack {x,y} \right\rbrack}} = {\sum\limits_{n_{1} = {- \infty}}^{\infty}{\sum\limits_{n_{2} = {- \infty}}^{\infty}{{f\left\lbrack {{x - n_{1}},{y - n_{2}}} \right\rbrack} \cdot {g\left\lbrack {n_{1},n_{2}} \right\rbrack}}}}}\end{matrix} & (3)\end{matrix}$

In equation (3), f*g represents the third function, “f” and “g”represent the two functions that the convolution procedure 316 is beingperformed on, x and y represent variables that the functions “f” and “g”depend on, and “n1” and “n2” represent an amount of shift. In oneembodiment, the convolution neural network module 310 can implement theconvolution procedure 316 using hardware, software, or a combinationthereof. For example, the convolution neural network module 310 caninclude processors, active and passive components, circuitry, controllogic, or a combination thereof implementing the convolution procedure316.

In one embodiment, the auditory spectrogram 312 can represent one of thefunctions in equation (3), for example “f” In one embodiment, theconvolution neural network module 310 can perform the convolutionprocedure 316 on the auditory spectrogram 312 by convoluting theauditory spectrogram 312 with a further function, for example one ormore convolution functions 318, which can represent “g” in equation (3).In one embodiment, the convolution functions 318 can be one or morematrices of size N×N, where N represents an integer. For example, theconvolution functions 318 can be a 3×3 matrix, 5×5 matrix, 10×10 matrix,or any other sized matrix. In one embodiment, the convolution functions318 can further have one or more values represented by integers or wholenumbers for their N×N elements. The one or more values can be the sameor different for each N×N element. The one or more values can further bethe same or different for each of the convolution functions 318.

In one embodiment, the one or more values of the convolution functions318 can be obtained through a training process in which the one or morevalues are obtained through back propagation of machine learned traineddata. The machine learned trained data can be, for example previousauditory spectrograms 312 representing known respiratory abnormalities,for example sounds, noises, auditory tones, or a combination thereofcaused by a cough, a crackle, a wheeze, an asthma attack or otherrespiratory event or respiratory condition. The back propagation cangenerate the one or more values of the convolution functions 318 bycalculating and attempting to reconstruct a known output, such aspreviously analyzed auditory spectrograms 312 representing knownrespiratory abnormalities, and determining what values the convolutionfunctions 318 must have to reconstruct the previously analyzed auditoryspectrograms 312 representing known respiratory abnormalities. As such,the one or more values of the convolution functions 318 can be obtainedand can be optimized to extract one or more features, such as curves,peaks, valleys, shapes, lines, or colors represented in the previouslyanalyzed auditory spectrogram 312 which can be used to reconstruct thepreviously analyzed auditory spectrograms 312. Once determined, the oneor more values of the convolution functions 318 can further be used tosubsequently recognize the same features in further auditoryspectrograms 312, such that the convolution functions 318 can be used torecognize respiratory abnormalities from auditory signals 262,accelerometer data 272, or a combination thereof detected by the digitalstethoscope 110.

Continuing with the example, once the convolution functions 318 andtheir values are generated, the convolution neural network module 310can perform the convolution procedure 316 on the auditory spectrogram312 using the convolution functions 318 by performing the convolutionprocedure 316 over N×N frames of the auditory spectrogram 312 andmultiplying the values of the N×N frame of the auditory spectrogram 312with the values of the convolution functions 318. As a result, a furthermatrix is generated with one or more convolution values 320 representingthe extracted features of the auditory spectrogram 312 for that N×Nframe. The convolution values 320 can be integers or whole numbers thatgive quantified values to the features of the auditory spectrogram 312for a particular N×N frame. By way of example, in areas where theauditory spectrogram 312 has a value greater than 0, the convolutionvalues 320 generated can be positive integers assuming the one or morevalues of the convolution functions 318 are also non-zero. However, inareas where the auditory spectrogram 312 has a zero value, theconvolution values 320 generated can be zero.

In one embodiment, the convolution procedure 316 can be performedrepeatedly to successively compress the auditory spectrogram 312 intoone or more convolution matrices 322, each having less rows and columnsthan the previous convolution matrices 322 generated as a result of theconvolution procedure 316. In one embodiment, the generation of thesubsequent convolution matrices 322 can depend on the values of theprevious convolution matrices 322. For example, in one embodiment, iftwo convolution matrices 322 are generated as a result of theconvolution procedure 316, the first can have the size M×M and thesecond can have the size T×T, wherein the second can have itsconvolution values 320 depend on the first, and where M>T. In oneembodiment, the convolution matrices 322 can be stored in the firstDRAM, wherein the convolution matrices 322 can be used in successivecalculation of further convolution matrices 322.

In one embodiment, as a result of performing the convolution procedure316 successively, a convolution vector 324 can be generated containingconvolution values 320. For example, in one embodiment, the convolutionvector 324 can be an M×1 vector, where M is an integer, such as a 20×1vector, containing convolution values 320. In one embodiment, theconvolution vector 324 can be used as the input to a machine learningprocess, for example the neural network 326, which can be used todetermine whether the convolution vector 324 representing a compressedversion of the auditory spectrogram 312, accelerometer data 272, or acombination thereof indicates a respiratory abnormality. The neuralnetwork 326 refers to an interconnected group of artificial neurons thatuses a mathematical or computational model for information processingbased on a connectionistic approach to computation.

The neural network 326 can use the weights 276 of FIG. 2C and one ormore biases 334 for each of its nodes or neurons which can be used todetermine whether the auditory signals 262 represent a respiratoryabnormality. Biases 334 refer to values, variables, parameters, or acombination thereof which enables the neural network 326 to better fitthe data to assist the neural network 326 to learn patterns. In oneembodiment, the weights 276 and biases 334 can be integers or wholenumbers. The weights 276 and biases 334 can connect nodes or neurons ofthe neural network 326, and can determine the strength of eachconnection between each node or neuron such that the higher the valuefor the weights 276 and biases 334 for a given connection, the more theconnection will affect the overall computations and outcomes of theneural network 326. In one embodiment, the weights 276 and biases 334can be determined and obtained in a similar process of back propagationof training data similar to what was described above with respect to theone or more values of the convolution functions 318. For the purposes ofdiscussion, it is assumed that the weights 276 and biases 334 arealready known and optimized for detecting the respiratory abnormality.

Continuing with the example, in one embodiment, the neural network 326can further have a confidence level 328 associated with its output. Theconfidence level 328 can be a value, variable, parameter, or acombination thereof which, if the output of the neural network 326exceeds, the neural network 326 can determine that the output can beclassified as either “normal” or “abnormal.” By way of example, if theconfidence level 328 is set to for example “90%,” any output of theneural network 326 indicating that the neural network 326 believes theauditory signals 262, accelerometer data 272, or a combination thereofdetected is greater than or equal to 90% “normal” or “abnormal” will beaccepted as a valid result and classified accordingly by the neuralnetwork 326. In one embodiment, if the confidence level 328 is met orexceeded as a result of the processing by the neural network 326, thedigital stethoscope 110 can generate the classification value 266indicating a “normal” or “abnormal” classification, and display anindicator based on the classification value 266, such as a red or greencolored light, or a message indicating that the auditory signals 262 are“abnormal” or “normal.”

It has been discovered that the use of the convolutional neural networkmodule 310 described above allows the digital stethoscope 110 togenerate the classification value 266 and display the indicator in realtime when used in conjunction with the architecture of the digitalstethoscope 110 because it uses less computing resources, such asprocessing power and memory as opposed to conventional methods ofdetecting a respiratory abnormality. Real time refers to the instancewhere the classification value 266 and the indicator can be displayedalmost instantly or within milliseconds of receipt of the auditorysignals 262, the accelerometer data 272, or a combination thereof by thedigital stethoscope 110.

It has been further discovered that the ability to generate theclassification value 266 and display the indicator in real timesignificantly improves the current state of the art by allowing patientsor users of the system 100 to be notified of a respiratory abnormalityinstantaneously and seek treatment immediately. It has been furtherdiscovered that the ability of patients or users of the system 100 to benotified of the respiratory abnormality in real time can result in theability to potentially save more patient lives by allowing patients toknow that they need to seek medical treatment rapidly if a respiratoryabnormality is detected.

Referring now to FIG. 4A, therein is shown an exemplary depiction of thefunctioning of the receiver module 304 and the subtraction module 306 inan embodiment of the present invention. In the embodiment shown in FIG.4A, once the auditory signals 262 are received, by the receiver module304, the auditory signals 262 can be converted from time functions 302to frequency domain functions 330 as discussed previously with respectto FIG. 3 . By way of example, FIG. 4A shows two frequency domainfunctions y(w) and N(w). In the embodiment shown in FIG. 4 , y(w) canrepresent the frequency domain function 330 of auditory signals 262obtained by the receiver module 304 that should be kept by the digitalstethoscope 110 while N(w) can represent the frequency domain function330 of auditory signals 262 obtained by the receiver module 304 thatshould not be kept by the digital stethoscope 110. Further, y(w) andN(w) are shown as containing one or more frequencies as depicted by afirst frequency band set 402 and a second frequency band set 404. Thefirst frequency band set 402 and the second frequency band set 404represent the different frequencies contained within y(w) and N(w) andcan be obtained by converting the time functions 302 to the frequencydomain functions 330. For example, in one embodiment, boxes labeled as“1” of the first frequency band set 402 or the second frequency band set404 can represent an audible portion of y(w) or N(w) that has a highfrequency, while boxes labeled as “4” of the first frequency band set402 or the second frequency band set 404 can represent an audibleportion of y(w) or N(w) that has a low frequency. In one embodiment,y(w) and N(w) can be passed to the subtraction module 306 so that thesubtraction module 306 can filter out any unwanted noise portions, forexample all or some of N(w) according to equation (2) above for each ofthe first frequency band set 402 and the second frequency band set 404.

Referring now to FIG. 4B, therein is shown an exemplary depiction of thefunctioning of the filter module 308 and the generation of the auditoryspectrogram 312 in an embodiment of the present invention. In theembodiment shown in FIG. 4B, the filtered frequency domain functions 330of the auditory signals 262 is shown as being passed to the filtermodule 308. The filter module 308 is further shown with the cochlearfilters 332. In one embodiment, the filtered frequency domain functions330 of the auditory signals 262 can pass through the cochlear filters332. As a result of passing the filtered frequency domain functions 330of the auditory signals 262 through the cochlear filters 332, theauditory spectrogram 312 is generated. The auditory spectrogram 312 canbe generated by, for example plotting the frequencies filtered by thecochlear filters 332 on a plot showing time versus frequency. In oneembodiment, the auditory spectrogram 312 can further indicate thefrequency bands 336 of the auditory signals 262.

Referring now to FIG. 4C, therein is shown an exemplary depiction of thefunctioning of the convolution neural network module 310 in anembodiment of the present invention. In the embodiment shown in FIG. 4C,the auditory spectrogram 312 is shown. Further, the convolutionfunctions 318 are shown as being represented by the one or more boxesrepresenting matrices of size N×N. The convolution neural network module310 can perform the convolution procedure 316 by convoluting theauditory spectrogram 312 with the convolution functions 318 by shiftingthe convolution functions 318 over the auditory spectrogram 312 andmultiplying the values for N×N frames of the auditory spectrogram 312with the values of the convolution functions 318. As a result, theconvolution matrices 322 are generated with convolution values 320. FIG.4C further depicts the generation of the subsequent convolution matrices322. In one embodiment, the subsequent convolution matrices 322 can begenerated by for example a pooling process 406, in which subsequentconvolution matrices 322 are generated by reducing the dimensions of theprevious convolution matrices 322 generated.

By way of example, a first convolution matrix 408 can be generated as aresult of the convolution procedure 316. In one embodiment, a secondconvolution matrix 410 can subsequently be generated based on the firstconvolution matrix 408 by the pooling process 406, by for example,taking groups of sub-elements of the first convolution matrix 408 andgenerating the second convolution matrix 410 based on the sub-elements.For example, if the first convolution matrix 408 is an M×M sized matrix,the first convolution matrix 408 can be broken down into sub-matrices,for example R×R sized sub-matrices, and a value chosen from the R×Rsub-matrix can be taken as one of the convolution values 320 of thesecond convolution matrix 410. The pooling process 406 can be done in avariety of ways, including max pooling in which the maximum value forthe R×R sub-matrix is taken as one of the convolution values 320 of thesecond convolution matrix 410. In another embodiment, the poolingprocess 406 can be done by average pooling in which the values of theR×R sub-matrix are averaged and the average is taken as one of theconvolution values 320 of the second convolution matrix 410.

In one embodiment, the pooling process 406 can be done repeatedly tosuccessively compress the auditory spectrogram 312 and generate theconvolution vector 324. The convolution vector 324 can then be input tothe neural network 326 which can use the weights 276 and biases 334 todetermine whether the convolution vector 324 represents a respiratoryabnormality by classifying the convolution values 320 associated withthe convolution vector 324 as being “normal” or “abnormal.”

In one embodiment, the second control flow 300 described above canfurther be used to make other classifications based on the auditorysignals 262 detected by the digital stethoscope 110. For example, inaddition to detecting a respiratory abnormality and making aclassification of “normal” or “abnormal,” the digital stethoscope 110can classify the auditory signals 262 as a particular type of sound,noise, acoustic tone, or a combination thereof using the same processesdescribed above with respect to the control flow 300. For example, inone embodiment, the digital stethoscope 110 using the second controlflow 300, can classify the auditory signals 262 as a wheeze, a crackle,or a cough based on the convolutional neural network module 310 havingits convolution functions 318 and neural network 326 trained torecognize a wheeze, a crackle, or a cough based on training with machinelearned trained data representing known sounds, noises, auditory tones,or a combination thereof caused by a cough, a crackle, a wheeze, or acombination thereof. As such, the classifications such as “cough” or “nocough,” “crackle” or “no crackle,” or “wheeze” or “no wheeze” can begenerated.

In one embodiment, the coughs, crackles, or wheezes detected can besaved in the configuration file 264 as the cough count 274 and can beused to predict a future respiratory event or respiratory condition. Forexample, if the cough count 274 is known and exceeds a certainthreshold, for example, is greater than the average number of coughs forthe patient within a certain time period, and the digital stethoscope110 further detects that the coughs indicate a respiratory abnormality,the digital stethoscope 110 can, for example, further generate aparameter, value, or variable indicating that a respiratory event orrespiratory condition is likely in the future and save the informationin the configuration file 264, which can further be shared with the basestation 118, the remote server 242, or a combination thereof to alertthe patient or a user of the system 100 of the likelihood of therespiratory event or respiratory condition. In one embodiment, theprediction of the respiratory event or respiratory condition can furtherbe aided by the use of machine learning processes, for example a longshort term memory (LSTM) network model. The LSTM network model will bediscussed further below.

Forecasting Using an LSTM

Referring now to FIG. 5A, therein is shown an exemplary feature map 500used to predict a respiratory event or respiratory condition in thefuture in an embodiment of the present invention. The feature map 500 isa chart, image, matrix, plot, or other visual depiction showing one ormore map features 512 versus time. In one embodiment, the feature map500 can be generated by the digital stethoscope 110, the base station118, the remote server 242, or a combination thereof. The map features512 refer to parameters, variables, or a combination thereof that areused to predict a respiratory event or respiratory condition in thefuture. The map features 512 describe one or more conditions at aparticular time or over a period of time that can affect the onset of arespiratory event or respiratory condition.

For example, in one embodiment, the map features 512 can include,environmental data, data regarding the patient, or a combinationthereof, indicating an environmental condition at a particular time orover a period of time affecting the onset of a respiratory event orrespiratory condition, or patient data, including data regarding coughs,crackles, wheezes, or a combination thereof, indicating the onset of arespiratory event or respiratory condition. For example, the mapfeatures 512 can include environmental data indicating how muchpollution is in the air at a particular time, how much pollen is in theair at a particular time, the air quality at a particular time, or caninclude data regarding the patient, such as data regarding whether thepatient is coughing or wheezing at a certain time, the severity of thepatient's cough, crackle, or wheeze at a certain time, how many times apatient is coughing or wheezing over a period of time, or a combinationthereof. As such, the map features 512 can include a pollution leveldata 502, a weather data 504, a severity score 506 indicating how severea patient's cough, crackle, wheeze, respiratory event, or respiratorycondition is at a given time, the cough count 274, a pollen data 510, ora combination thereof.

In one embodiment the severity score 506 can be categorized indicatingthe level of severity of the patient's cough, crackle, wheeze,respiratory event, or respiratory condition at a given time. In oneembodiment, the categorization can be, for example on a scale of 1-6where, 1 indicates no or non-present severity, 2 indicates intermittentseverity, 3 indicates mild severity, 4 indicates moderate severity, 5indicates severe or high severity, and 6 indicates acute severity inwhich the patient should get immediate medical attention. In oneembodiment, the severity score 506 can be determined by the digitalstethoscope 110, the base station 118, the remote server 242, or acombination thereof by analyzing the auditory signals 262 received fromthe digital stethoscope 110, and analyzing the auditory signals 262pursuant to the second control flow 300 or equivalent control flow todetermine the category of the severity score 506, wherein the neuralnetwork 326 or a further neural network can be used to determine thecategory of the severity score 506.

Continuing with the example, in one embodiment, the map features 512 canbe obtained from the digital stethoscope 110, the base station 118, or acombination thereof, and the components thereof including for examplethe microphones 106, the air quality sensors 116, or a combinationthereof. In one embodiment, the map features 512, the feature map 500,or a combination thereof can be used as an input to a machine learningprocess that can be used to predict a respiratory event or respiratorycondition in the future. For example, the map features 512, the featuremap 500, or a combination thereof can be the input to an LSTM process514 to predict the respiratory event or respiratory condition in thefuture. In one embodiment, more than one feature map 500 can be theinput to the LSTM process 514 to predict the respiratory event orrespiratory condition in the future. The LSTM process 514 refers toartificial recurrent neural network (RNN) architecture which usesfeedback to make predictions about future events.

Referring now to FIG. 5B, therein is shown an exemplary depiction of theLSTM process 514 used to predict a third respiratory event 520 occurringfor the patient at a future time in an embodiment of the presentinvention. The LSTM process 514 of FIG. 5B can work in the followingmanner. In the embodiment of FIG. 5B, one or more time buckets 518 areshown as represented by “n−1,” “n,” and “n+1,” wherein the time buckets518 contain information regarding the map features 512 at a particularinstance of time or over a period of time. In one embodiment, the timebuckets 518 can be implemented as data structures, variables,parameters, or a combination thereof as nodes in the LSTM process 514.In one embodiment, each instance of the time buckets 518 can bepredicted by implementing a look back feature 516, which references themap features 512 of previous time buckets 518 and performs a probabilitycalculation to determine the likelihood that future time buckets 518will have map features 512 exhibiting certain characteristics or havingcertain values. For example, “n” can be predicted by referencing the mapfeatures 512 of “n−1,” and “n+1” can further be predicted by referencingthe map features 512 of “n−1,” “n,” and so forth.

In one embodiment, the prediction is done by comparing the map features512 of past time buckets 518 to one another and finding one or morepatterns or correlations between the map features 512. The patterns orcorrelations can be, for example, map features 512 indicating a highprobability of exhibiting a certain characteristic or having a certainvalue based on one or more previous map features 512 exhibiting certaincharacteristics or having certain values. By way of example, if it isdetermined that when pollution is high and a patient has a certainnumber of coughs during the period when the pollution is high, there isa high probability that the severity scores 506 of a patient's coughs,crackles, or wheezes will be “5”-severe or “6”-acute, the LSTM process514 can look for that pattern or correlation when comparing the mapfeatures 512 of past time buckets 518, and if that pattern orcorrelation is detected, can predict that in instances where a pollutionlevel is getting to a high level and the patient has a certain number ofcoughs building up to a certain threshold, the severity score 506 ofthose coughs is likely to be “5”-severe or “6”-acute in the near future.The example is merely exemplary and not meant to be limiting.

In one embodiment, the LSTM process 514 can contain weights 276associated with the look back feature 516, similar to the weights 276described with respect to FIG. 2C. The weights 276 can be used todetermine the strength of each connection between the time buckets 518such that the higher the value for the weights 276 for a given look backfeature 516, the more the map features 512 of the particular timebuckets 518 associated with the look back feature 516 will affect theoverall computations and predictions of the LSTM process 514. In oneembodiment, the weights 276 can be determined and obtained in a similarprocess of back propagation of training data similar to what wasdescribed above with respect to the one or more values of theconvolution functions 318. For the purposes of discussion, it is assumedthat the weights 276 are already known and optimized for predicting mapfeatures 512 for future time buckets 518 based on a history of known mapfeatures 512 associated with known respiratory events and respiratoryconditions.

In one embodiment, the predictions made by the LSTM process 514 canfurther be output to the digital stethoscope 110, the base station 118,the remote server 242, or a combination thereof. For example, if theLSTM process 514 predicts that a patient will have a severe or acuterespiratory event or respiratory condition in the near future, an outputcan be generated indicating the likelihood of the respiratory event orrespiratory condition to, for example, the display unit 102 of thedigital stethoscope 110 indicating the prediction.

It has been discovered that the use of the LSTM process 514 describedabove allows the digital stethoscope 110, the base station 118, or theremote server 242 to generate accurate predictions for the likelihood ofoccurrence of respiratory events or respiratory conditions. It has beenfurther discovered that the ability to generate predictions regardingrespiratory events or respiratory conditions significantly improves thecurrent state of the art by allowing patients or users of the system 100to be notified of respiratory events or respiratory conditions beforethey occur and allow patients or users of the system 100 to seektreatment in advance of a medical emergency. It has been furtherdiscovered that the ability of patients or users of the system 100 to benotified of the respiratory events or respiratory conditions can resultin the ability to potentially save more patient lives by allowingpatients to know that they need to seek medical treatment rapidly if arespiratory event or respiratory condition is predicted.

Convolutional Auto-Encoder to Predict Future Respiratory Noises

Referring now to FIG. 6 , therein is shown an exemplary third controlflow 600 for forecasting characteristics of a future respiratory eventor respiratory condition in an embodiment of the present invention. Forbrevity of description, in this embodiment, the third control flow 600will be described as being performed using the base station 118. Thisdescription is merely exemplary and not meant to be limiting. In otherembodiments, the third control flow 600 can be performed using thedigital stethoscope 110, the remote server 242, or a combinationthereof.

The third control flow 600, can enable forecasting characteristics of afuture respiratory event or respiratory condition by using one or morefeature spaces 602 generated based on the auditory signals 262, whereinthe feature spaces 602 are a three dimensional map or representation ofthe auditory signals 262. In one embodiment, the feature spaces 602 canbe generated by the base station 118, for example, by the second FPGA244 after receiving the auditory signals 262. The feature spaces 602 canbe a chart, image, matrix, plot, or other visual depiction representingthe auditory signals 262 as a three dimensional map. The feature spaces602 can have one or more variables with values representingcharacteristics of the auditory signals 262. For example, in oneembodiment, the variables can include a frequency associated with theauditory signals 262 at a particular time, a rate representing a map ofauditory nerve firing rate indicated by the auditory signals 262 at aparticular time, and a severity representing the magnitude of theauditory signal 262 at a particular time. The feature spaces 602 can begenerated using the same techniques as generating the auditoryspectrogram 312, where the auditory signals 262 can be received,converted into the frequency domain functions 330, and filtered usingthe cochlear model 314 to obtain the variables of the feature spaces602. Once the feature spaces 602 are generated, the third control flow600 can be used to forecast characteristics of a future respiratoryevent or respiratory condition using the feature spaces 602.

The third control flow 600 can be implemented with modules andsub-modules, the components of the base station 118, or a combinationthereof. In one embodiment, the third control flow 600 can include theconvolution neural network module 310, a reverse convolution module 604,a validation module 606, and a long short term memory module 610. In oneembodiment, the convolution neural network module 310 can couple to thereverse convolution module 604. The convolution neural network module310 can further couple to the long short term memory module 610 via astorage 608. The reverse convolution module 604 can couple to thevalidation module 606.

In one embodiment, the convolution neural network module 310, aspreviously described with respect to FIG. 3 , can receive the featurespaces 602 and perform the convolution procedure 316 on the featurespaces 602 to generate a convolution vector 324 for each of the featurespaces 602. In one embodiment, once the convolution vector 324 isgenerated, the convolution vector 324 can be saved in a storage 608 forlater use by the long short term memory module 610. The storage 608 canbe the second storage unit 248 or a further storage unit external to thebase station 118, such as an external database, hard drive, volatilememory, nonvolatile memory, or a combination thereof.

In one embodiment, once the convolution neural network 310 has completedthe convolution procedure 316 for each of the feature spaces 602, eachconvolution vector 324 and control can further be passed to the reverseconvolution module 604. The reverse convolution module 604 enables thereverse process of the convolution neural network module 310, such thatthe reverse convolution module 604 can decode and reconstruct thefeature spaces 602 by, for example, reversing the computations done bythe convolution neural network module 310 to generate the feature spaces602 from the convolution vector 320. In one embodiment, the reversing ofthe convolution vector 324 can be used to forecast characteristics of afuture respiratory event or respiratory condition as will be describedbelow. The reverse convolution module 604 can perform the reverseprocess of the convolution neural network module 310, by for exampleperforming a reverse of the convolution procedure 316 by undoing all thecalculations performed by the convolution procedure 316.

In one embodiment, once the reverse convolution module 604 reconstructsthe feature spaces 602, it can pass the reconstructed feature spaces 602and control to the validation module 606. The validation module 606enables the validation of the of the computations of the reverseconvolution module 604 by for example comparing the reconstructedfeature spaces 602 with the feature spaces 602 received by theconvolution neural network module 310. For example, in one embodiment,the comparison can include comparing the values of the variables, forexample the frequency, rate, and scale of each the reconstructed featurespaces 602 with the same variables for feature spaces 602 received bythe convolution neural network module 310 and determining whether theyare equal or within an error tolerance, such that the two can beconsidered equivalent. If the validation module 606 determines that thevalues are equivalent, the computations of the reverse convolutionmodule 310 can be considered valid such that reverse convolution module604 can later be used to forecast characteristics of a futurerespiratory event or respiratory condition.

In one embodiment, once the validation module 606 has validated thecomputations of the reverse convolution module 604, control can pass tothe long short term memory module 610. The long short term memory module610 can enable and implement the LSTM process 514 as described withrespect to FIG. 5B. In one embodiment, the long short term memory module610 can retrieve each saved convolution vector 324 from the storage 608and perform the LSTM process 514 using each convolution vector 324 in asimilar fashion as was described with the time buckets 518 of FIG. 5B.By way of example, the long short term memory module 610 can replace thetime buckets 518 with each convolution vector 324 and predict a futureconvolution vector 324 using the values of each convolution vector 324.

In one embodiment, the long short term memory module 610 can look forpatterns as described with respect to FIG. 5B using a look back feature516 and weights 276 trained based on a history of respiratory events orrespiratory conditions, but with respect to the convolution vectors 324such that the long short term memory module 610 can forecastcharacteristics of a future respiratory event or respiratory conditionbased on each previous convolution vector 324.

By way of example, if the long short term memory module 610 determinesthat each previous convolution vector 324 indicates a trend towardsfeature spaces 602 for a known respiratory event or respiratorycondition, the long short term memory module 610 can forecastcharacteristics of future feature spaces 602 such that the long shortterm memory module 610 can predict or forecast particular feature spaces602 indicating a particular respiratory event or respiratory condition.In one embodiment, the forecasting can be done by generating a predictedconvolution vector 324 with predicted values indicating the predicted orforecasted characteristics of the forecast feature spaces 602. Thepredicted or forecast feature spaces 602 can further be generated bydecoding or reversing the predicted convolution vector 324 via thereverse convolution procedure module 604.

In one embodiment, the predicted or forecast feature spaces 602 canfurther be output to the digital stethoscope 110, the base station 118,the remote server 242, or a combination thereof. For example, if thelong short term memory module 610 forecasts that a patient will havefeature spaces 602 indicating a particular respiratory event orrespiratory condition in the near future, an output can be generatedindicating the forecast or predicted feature spaces 602 to, for example,the display unit 102 of the digital stethoscope 110 and an alert can besent to the patient or user of the system 100 indicating theforecasting.

It has been discovered that the use of the third control flow 600described above allows the digital stethoscope 110, the base station118, or the remote server 242 to generate accurate predictions for thelikelihood of occurrence of respiratory events or respiratory conditionswithout the need to know about environmental conditions at a particulartime or over a period of time affecting the onset of a respiratory eventor respiratory condition and can generate forecasts of characteristicsof respiratory events or respiratory conditions based on the auditorysignals 262. It has been further discovered that the ability to generatepredictions regarding respiratory events or respiratory conditionssignificantly improves the current state of the art by allowing patientsor users of the system 100 to be notified of a respiratory event orrespiratory conditions before they occur and allow patients or users ofthe system 100 to seek treatment in advance of a medical emergency. Ithas been further discovered that the ability of patients or users of thesystem 100 to be notified of the respiratory events or respiratoryconditions can result in the ability to potentially save more patientlives by allowing patients to know that they need to seek medicaltreatment rapidly if a respiratory event or respiratory condition ispredicted.

Referring now to FIG. 7 , therein is shown an exemplary depiction of thethird control flow 600 for forecasting characteristics of a futurerespiratory event or respiratory condition in an embodiment of thepresent invention. In the embodiment of FIG. 7 , two feature spaces 602are shown, as indicated by 602A and 602B representing a frequency, rate,and scale of the auditory signals 262 at different times T(1) and T(2).FIG. 7 further shows 602A and 602B undergoing the convolution procedure316, as indicated by 704A and 704B. As a result, one or more convolutionmatrix spaces 702 are generated, as indicated by 702A, 702B, 702C, and702D. 702A, 702B, 702C, and 702D, similar to the convolution matrices322, represent spaces generated from doing the convolution procedure 316on three dimensional feature spaces 602, such that they represent threedimensional matrix spaces. FIG. 7 further shows the convolution matrixspaces 702 being compressed from one iteration of the convolutionprocedure 316 to the next such that the convolution matrix spaces 702 ofa subsequent iteration are smaller than the convolution matrix spaces702 of the previous iteration, similar to what was described withrespect to FIGS. 3 and 4C, until the convolution matrix spaces 702 canbe formed into a convolution vector 324 associated with each of thefeature spaces 602. The convolution vector 324 associated with each ofthe feature spaces 602 are represented as 324A and 324B. FIG. 7 ,further shows the reversing or decoding of the convolution procedure316, as indicated by 708A and 708B to generate one or more reverseconvolution matrix spaces 704, as indicated by 704A and 704B. Thereverse convolution matrix spaces 704 represent three dimensional matrixspaces generated from performing the reverse convolution procedure 316on 324A and 324B. FIG. 7 further shows one or more reconstructed featurespaces 710, as indicated by 710A and 710B, generated based on performingthe reversing of the convolution procedure 316 on the reverseconvolution matrix spaces 704.

In the embodiment shown in FIG. 7 , each of 324A and 324B can further beused to forecast characteristics of a future respiratory event orrespiratory condition based on having the LSTM process 514, as done bythe long short term memory module 610, performed using 324A and 324B asthe inputs of the LSTM process 514. For example, FIG. 7 shows the LSTMprocess 514 performed, as indicated by 514A and 514B, such that aprediction vector 324N, representing a predicted convolution vector 324is generated for a future time, indicated by T(N). The prediction vector324N can have a reverse convolution procedure 316 performed on it, asindicated by 708N to generate a predicted feature space 710Nrepresenting the forecasted characteristics of a future respiratoryevent or respiratory condition.

The system 100 has been described with module functions or order as anexample. The system 100 can partition the modules differently or orderthe modules differently. For example, the first software 224, the secondsoftware 252, or a combination thereof, can include the modules for thesystem 100. As a specific example, the first software 224, the secondsoftware 252, or a combination thereof can include the receiver module304, the subtraction module 306, the filter module 308, the convolutionneural network module 310, the reverse convolution module 604, thevalidation module 606, the long short term memory module 610, andassociated sub-modules included therein.

The first control unit 210, the second control unit 236, or acombination thereof, can execute the first software 224, the secondsoftware 252, or a combination thereof, to operate the modules. Forexample, the first control unit 210, the second control unit 236, or acombination thereof, can execute the first software 224, the secondsoftware 252, or a combination thereof, to implement the receiver module304, the subtraction module 306, the filter module 308, the convolutionneural network module 310, the reverse convolution module 604, thevalidation module 606, the long short term memory module 610, andassociated sub-modules included therein.

The modules described in this application can be implemented asinstructions stored on a non-transitory computer readable medium to beexecuted by the first control unit 210, the second control unit 236, ora combination thereof. The non-transitory computer readable medium caninclude the first storage unit 218, the second storage unit 248, or acombination thereof. The non-transitory computer readable medium caninclude non-volatile memory, such as a hard disk drive, non-volatilerandom access memory (NVRAM), solid-state storage device (SSD), compactdisk (CD), digital video disk (DVD), or universal serial bus (USB) flashmemory devices. The non-transitory computer readable medium can beintegrated as a part of the system 100 or installed as a removableportion of the system 100.

Exemplary Methods

Referring now to FIG. 8 , therein is shown an exemplary method 800 ofoperating the system 100 in an embodiment of the present invention. Themethod 800 includes: receiving, from a microphone, an auditory signal asshown in box 802; generating an auditory spectrogram based on theauditory signal as shown in box 804; performing a convolution procedureon the auditory spectrogram to generate one or more convolution values,wherein the convolution values represent a compressed version of aportion of the auditory spectrogram, and wherein the convolutionprocedure is trained to generate one or more features for detecting therespiratory abnormality as shown in box 806; applying one or moreweights to the convolution values, wherein the weights are trained fordetection of the respiratory abnormality as shown in box 808; generatinga classification value based on applying the weights to the convolutionvalues, wherein the classification value indicates whether the auditorysignal includes the respiratory abnormality as shown in box 810; andoutputting the classification value as shown in box 812.

Referring now to FIG. 9 , therein is shown a further exemplary method900 of operating the system 100 in an embodiment of the presentinvention. The method 900 includes: receiving, over a period of timefrom a microphone of a digital stethoscope, an auditory signal as shownin box 902; generating an auditory spectrogram based on the auditorysignal as shown in box 904; analyzing, with a control unit of thedigital stethoscope, the auditory spectrogram to determine whether theauditory signal represents a cough as shown in box 906; tracking, withthe control unit of the digital stethoscope, the cough in a cough log asshown in box 908; and transmitting based on the cough log, with acommunication unit of the digital stethoscope to a cloud-based service,a message indicating a number of coughs tracked over the period of timefor storage on a remote server as shown in box 910.

Referring now to FIG. 10 , therein is shown a further exemplary method1000 of operating the system 100 in an embodiment of the presentinvention. The method 1000 includes: receiving, from a microphone of adigital stethoscope, a first noise signal at a first time as shown inbox 1002; analyzing the first noise signal to determine a first severityscore indicating how severe a first respiratory event of a patientcaptured in the first noise signal is as shown in box 1004; receiving,from the microphone of the digital stethoscope, a second noise signalcaptured at a second time as shown in box 1006; analyzing the secondnoise signal to determine a second severity score indicating how severea second respiratory event of the patient captured in the second noisesignal is as shown in box 1008; generating a prediction indicating alikelihood of a third respiratory event occurring for the patient at afuture time based on applying a first weight to the first severity scoreand a second weight to the second severity score, wherein the first andsecond weights are trained based on a history of respiratory events asshown in box 1010; outputting the prediction as shown in box 1012.

Referring now to FIG. 11 , therein is shown a further exemplary method1100 of operating the system 100 in an embodiment of the presentinvention. The method 1100 includes: receiving, from a microphone of adigital stethoscope, a first noise signal capturing a first respiratoryevent at a first time as shown in box 1102; generating a first featurespace representing the first noise signal as shown in box 1104; encodingthe first feature space into a first convolution vector using aconvolution procedure as shown in box 1106; receiving, from themicrophone of the digital stethoscope, a second noise signal capturing asecond respiratory event at a second time as shown in box 1108;generating a second feature space representing the second noise signalas shown in box 1110; encoding the second feature space into a secondconvolution vector using the convolution procedure as shown in box 1112;determining a predicted convolution vector based on the first and secondfeature spaces as shown in box 1114; and decoding the predictedconvolution vector into a predicted feature space representing a soundmade by the future respiratory event as shown in box 1116.

Other Examples and Embodiments

Referring now to FIG. 12 , therein is shown exemplary spectrograms 1200representing breathing patterns in an embodiment of the presentinvention. The spectrograms 1200 of FIG. 12 , similar to what wasdescribed with respect to the auditory spectrogram 312 of FIG. 3 , aretwo dimensional visual representations of the spectrum of frequencies ofknown noises, from the patient's body, for example a cough, a crackle, awheeze, a breathing pattern, or a combination thereof. FIG. 12 shows twospectrograms 1200 representing two different breathing patterns, one fora normal breathing pattern 1202, indicating no respiratory abnormality,and one for a wheeze breathing pattern 1204, indicating a respiratoryabnormality associated with a wheeze. In one embodiment, the breathingpatterns and their representations in the form of the spectrograms 1200can be used as a part of the back propagation process, as described withrespect to FIGS. 3 and 5 , to train the weights 276 and biases 334 ofthe machine learning processes used by the system 100 which are used bythe system 100 to perform the detection of the respiratory abnormality,or to predict a respiratory event or respiratory condition in thefuture. For example, the spectrograms 1200 can be used to train theweights 276 and biases 334 used in the neural network 326, the LSTMprocess 514, or a combination thereof by for example using thespectrograms 1200 as examples of known breathing patterns which thesystem 100 can attempt to reconstruct through back propagation, and inthe process determine the values of the weights 276 and biases 334 toreconstruct the spectrograms 1200. The biases 334 and weights 276 cansubsequently be used to make future predictions about auditory signals262 and classify them, by for example, classifying them as “normal” or“abnormal” based on the processes previously described

Referring now to FIG. 13 , therein is shown an exemplary auditoryspectrogram 312 in an embodiment of the present invention. FIG. 13 showsa depiction of the auditory spectrogram 312 as described with respect toFIG. 3 . FIG. 13 further shows how the auditory spectrogram 312amplifies and sharpens the frequency components of an auditory signal262. For example, FIG. 13 , shows the frequency components of anauditory signal 262 as white marks. Further, FIG. 13 shows the magnitudeof the frequency components by indicating lighter shades of white forfrequency components that comprise a larger portion of the auditorysignal 262 for a given time.

Referring now to FIG. 14 , therein is shown an exemplary feature space602 in an embodiment of the present invention. FIG. 14 shows anexemplary frequency, rate, and scale of an auditory signals 262 atdifferent times plotted against one another other. For clarity and easeof description, the frequency, rate, and scale are plotted as twodimensional visual representations where each of the frequency, rate,and scale is plotted against only one of the other variables, despitebeing described in FIGS. 6 and 7 as a three dimensional map orrepresentation of the auditory signals 262. The combination of the plotsof FIG. 14 represents the feature space 602. FIG. 14 further shows threeplots 1402, 1404, and 1406. Plot 1402 represents a plot of the scale vs.rate. Plot 1404 represents a plot of frequency vs. rate. Plot 1406represents a plot of scale vs. frequency. As described in FIGS. 6 and 7, the plots can be constructed and reconstructed to make the predictionsregarding respiratory events or respiratory conditions in the future.

The above embodiments are described in sufficient detail to enable thoseskilled in the art to make and use the invention. It is to be understoodthat other embodiments would be evident based on the present disclosure,and that system, process, or mechanical changes may be made withoutdeparting from the scope of an embodiment of the present invention.

In the above description, numerous specific details are given to providea thorough understanding of the invention. However, it will be apparentthat the invention may be practiced without these specific details. Toavoid obscuring an embodiment of the present invention, some well-knowncircuits, system configurations, and process steps are not disclosed indetail.

The term “module” or “unit” referred to herein can include software,hardware, or a combination thereof in an embodiment of the presentinvention in accordance with the context in which the term is used. Forexample, the software can be machine code, firmware, embedded code, orapplication software. Also for example, the hardware can be circuitry, aprocessor, a microprocessor, a microcontroller, a special purposecomputer, an integrated circuit, integrated circuit cores, a pressuresensor, an inertial sensor, a microelectromechanical system (MEMS),passive devices, or a combination thereof. Further, if a module or unitis written in the system or apparatus claims section below, the moduleor unit is deemed to include hardware circuitry for the purpose and thescope of the system or apparatus claims.

The modules and units in the following description of the embodimentscan be coupled to one another as described or as shown. The coupling canbe direct or indirect, without or with intervening items between coupledmodules or units. The coupling can be by physical contact or bycommunication between modules or units.

The above detailed description and embodiments of the disclosed system100 are not intended to be exhaustive or to limit the disclosed system100 to the precise form disclosed above. While specific examples for thesystem 100 are described above for illustrative purposes, variousequivalent modifications are possible within the scope of the disclosedsystem 100, as those skilled in the relevant art will recognize. Forexample, while processes and methods are presented in a given order,alternative implementations may perform routines having steps, or employsystems having processes or methods, in a different order, and someprocesses or methods may be deleted, moved, added, subdivided, combined,or modified to provide alternative or sub-combinations. Each of theseprocesses or methods may be implemented in a variety of different ways.Also, while processes or methods are at times shown as being performedin series, these processes or blocks may instead be performed orimplemented in parallel, or may be performed at different times.

The resulting method, process, apparatus, device, product, and system iscost-effective, highly versatile, and accurate, and can be implementedby adapting components for ready, efficient, and economicalmanufacturing, application, and utilization. Another important aspect ofan embodiment of the present invention is that it valuably supports andservices the historical trend of reducing costs, simplifying systems,and increasing performance.

These and other valuable aspects of the embodiments of the presentinvention consequently further the state of the technology to at leastthe next level. While the invention has been described in conjunctionwith a specific best mode, it is to be understood that manyalternatives, modifications, and variations will be apparent to thoseskilled in the art in light of the descriptions herein. Accordingly, itis intended to embrace all such alternatives, modifications, andvariations that fall within the scope of the included claims. Allmatters set forth herein or shown in the accompanying drawings are to beinterpreted in an illustrative and non-limiting sense.

1. (canceled)
 2. A method of manufacture of a digital stethoscope fordetection of a respiratory abnormality, the method comprising: providingone or more microphones configured to detect an auditory signal;providing a field programmable gate array (FPGA) configured to: receive,from the one or more microphones, the auditory signal, generate anauditory spectrogram based on the auditory signal, perform a spatialconvolution procedure on the auditory spectrogram to generate one ormore convolution values, wherein the convolution values represent acompressed version of a matrix portion of the auditory spectrogram, andwherein the spatial convolution procedure is trained to generate one ormore features for detecting the respiratory abnormality, apply one ormore weights to the convolution values, wherein the weights are trainedfor detection of the respiratory abnormality, generate a classificationvalue based on applying the weights to the convolution values, whereinthe classification value indicates whether the auditory signal includesthe respiratory abnormality, and transmit the classification value to adisplay unit for outputting the classification value; providing thedisplay unit configured to display the classification value; couplingthe microphone to the FPGA; coupling the display unit to the FPGA; andintegrating the microphone, the FPGA, and the display into a housing. 3.The method of claim 2, further comprising: providing one or morebuttons; coupling the one or more buttons to the display unit; andintegrating the buttons into the housing.
 4. The method of claim 2,further comprising: providing a battery for powering the digitalstethoscope; coupling the battery to a battery board; coupling thebattery to the FPGA via the battery board; and integrating the batteryboard into the housing.
 5. The method of claim 2, wherein the batteryboard enables inductive charging of the battery.
 6. The method of claim2, further comprising: providing a printed circuit board (PCB); couplingthe FPGA to the PCB; and integrating the PCB into the housing.
 7. Themethod of claim 6, further comprising: providing an audio jack to outputnoise signals detected by the digital stethoscope; and coupling theaudio jack to the PCB.
 8. The method of claim 6, further comprising:providing a dynamic access random memory (DRAM) to store informationcaptured or processed by the digital stethoscope; integrating the DRAMinto the PCB; and coupling the DRAM to the FPGA via the PCB.
 9. Themethod of claim 8, further comprising: providing an accelerometer todetect movements of a body part by the digital stethoscope; integratingthe accelerometer into the PCB; and coupling the accelerometer to theDRAM via the PCB.
 10. The method of claim 2, further comprising:providing a glass lens to protect the display unit; coupling the glasslens to the display unit; and integrating the glass lens into thehousing.
 11. The method of claim 2, further comprising: providing a gripring for a user to hold the digital stethoscope; and coupling the gripring to an outer surface of the housing.
 12. The method of claim 2,further comprising: providing a display housing unit to hold the displayunit; coupling the display unit to the display housing unit; andintegrating the display housing unit into the housing.
 13. The method ofclaim 12, further comprising: providing a flex backing on which thedisplay housing unit sits; and integrating the flex backing into thehousing.
 14. The method of claim 13, further comprising: providing aflex assembly on which the flex backing sits; and integrating the flexassembly into the housing.
 15. The method of claim 14, furthercomprising: providing a retainer clip to hold the flex assembly inplace; and integrating the retainer clip into the housing.
 16. Themethod of claim 1, further comprising: providing a microphone assemblyon which the one or more microphones are housed; and integrating themicrophone assembly into the housing.
 17. The method of claim 1, furthercomprising: providing a diaphragm membrane forming the bottom surface ofthe digital stethoscope; and coupling the diaphragm membrane a bottom ofthe housing.
 18. The method of claim 17, further comprising: providing adiaphragm ring for providing a gripping surface for the bottom of thehousing; and coupling the diaphragm ring to the diaphragm membrane. 19.The method of claim 1, further comprising: providing a Bluetooth circuitfor communication with a base station; integrating the Bluetooth intothe housing.
 20. The method of claim 1, wherein the display unit is atouchscreen display.
 21. The method of claim 1, further comprising:arranging the one or more microphones in an array configuration.